1 Introduction

The aim of this paper is to present progress on the development of standardized measures of tenth grade students’ knowledge, attitudes and behaviours concerning sustainable development as those concepts are understood in the United Nations Educational, Scientific and Cultural Organization (UNESCO Education Sector 2006), the lead agency for the UN Decade for Education for Sustainable Development (DESD). The research was structured with two objectives in mind:

  1. (a)

    Using the standardized measures, establish a baseline of the knowledge, attitudes and behaviours of Manitoban tenth grade students towards sustainable development. With such a baseline in place, the Manitoba Department of Education might monitor evidence of changes in knowledge, attitudes and behaviours among students that might be correlated to current investments in education for sustainable development (ESD)

  2. (b)

    Provide the measures to other jurisdictions in Canada and internationally, which might be seeking to set similar baselines and monitor progress on ESD efforts.

1.1 Previous Work

In December 2002, the United Nations General Assembly (UNGA) adopted resolution 57/254 and designated UNESCO to lead the United Nations Decade of Education for Sustainable Development (DESD), spanning from 2005 to 2014. Focused on reorienting education on the long-term future of the economy, ecology and equity, the basic vision of the DESD is of a world in which everyone has the opportunity to benefit from education and learn the values, behaviours and lifestyles required for a sustainable future and for positive societal transformation. One of the overarching goals of the DESD is to reorient curricula, from preschool to university, reforming education as a vehicle of knowledge, values and actions needed to build a sustainable world. Many activities are underway, related to educational policy and programming for formal, non-formal and informal learning. However, a key challenge will be to assess whether in fact changes in behaviour are taking place as an outcome of the integration of sustainable development concepts and values into the learning process. Such changes can best be measured at the local and national levels; but in order to do so, baselines of current knowledge, attitudes and choices need to be established.

This is the second stage of research conducted in the province of Manitoba, Canada to develop and deploy standardized measures for assessing changes that might be correlated to investments in ESD. The questionnaires used in the survey reported here were refined versions of questionnaires used 2 years earlier. In the period from January to March 2008 two pilot surveys were undertaken in Manitoba. One used responses from 506 adults in a random sample of households from the whole province and one used a convenience sample of 294 students from grades 6–12 in four schools. Results of those pilots were published in a non-technical paper in Michalos et al. (2009) and in a technical paper in Michalos et al. (2011). Readers interested in a detailed account of the procedures used to design those earlier questionnaires may consult either of those publications.

Our basic hypothesis is that people’s behaviours are influenced primarily by their knowledge and attitudes, and in our earlier explorations we showed that this assumption was true for the student sample and partly true for the adult sample. For the latter, in the presence of respondents’ level of education as a predictor, which had a statistically significant impact on respondents’ behaviours, specific knowledge concerning sustainable development had no significant impact. In both cases, attitudes had more influence than knowledge on behaviours. Given our relative success with the relatively rough measures we developed for the earlier studies, we thought that further exploration was warranted to refine our measures, obtain a larger sample, and set a baseline. We were fortunate in obtaining the funding and other support necessary to continue our research.

1.2 Definitions

As we wrote earlier, there are a variety of ideas about the meaning of ‘sustainable development’ (SD) itself as well as ‘education for sustainable development’ (ESD). In his Review of Contexts and Structures for Education for Sustainable Development 2009, Wals (2009, p. 25) wrote:

Despite the many reports on ESD prompted by the Decade [of Education for Sustainable Development], its precise meaning continues to be the subject of worldwide debate. The regional synthesis reports that served as input for this mid-Decade review provide a variety of ESD definitions which share some common elements but are subtly different when addressing larger regional issues…At the beginning of the Decade, there was a greater perceived need to arrive at an agreed definition whereas now there appears more room for localization and contextualization. To this end, national and local debates are seen as crucial to develop the meaning of ESD.

ESD is supposed to be an umbrella concept covering several other UN initiatives and types of education including, for example, Environmental Education, Education For All, the Millennium Development Goals and the United Nations Literacy Decade (Wals 2009, pp. 8–10, 25–30). However, as one examines ESD initiatives around the world and even across the Canadian provinces and territories (CMEC 2010), it becomes clear that for many people there is little or no difference between ESD and Environmental Education. Articles 8 and 9 of the Bonn Declaration, which was accepted by the delegates at the UNESCO World Conference on Education for Sustainable Development in Bonn, Germany (31 March to 2 April, 2009) emphasize the richness of ESD.

8. ESD is based on values of justice, equity, tolerance, sufficiency and responsibility. It promotes gender equality, social cohesion and poverty reduction and emphasizes care, integrity and honesty, as articulated in the Earth Charter. ESD is underpinned by principles that support sustainable living, democracy and human well-being. Environmental protection and restoration, natural resource conservation and sustainable use, addressing unsustainable production and consumption patterns, and the creation of just and peaceful societies are also important principles underpinning ESD.

9. ESD emphasizes creative and critical approaches, long term thinking, innovation and empowerment for dealing with uncertainty, and for solving complex problems. ESD highlights the interdependence of environment, economy, society, and cultural diversity from local to global levels, and takes account of past, present and future” (UNESCO 2009).

Although the word ‘behaviour’ does not occur in these articles, people’s values, attitudes, skills, creative and critical approaches to solving problems can only be expressed in their actions or behaviour. Authors of the DESD Quarterly Highlights, Special COP15 Issue (December 2009, p. 1) emphasized that

Climate change issues need to be part of public awareness, learning and education for a sustainable future so that sustainable behaviours become daily habits. The DESD provides a framework for adaptation to and mitigation of climate change by enhancing and promoting active learning and innovative ways of framing climate change issues so that they make sense in the context of people’s everyday lives, helping to translate passive awareness into active concern and behaviour change…ESD is a key means to build a global lobby for effective action, showing people that their actions can contribute to lasting solutions.

In this report, as in our earlier report, we note again that the identification of specific behavioural changes required by individuals (especially 15 year olds) in the interest of sustainable development are not easily found. While we think we have made some progress beyond our earlier investigation, there is much more to be done on this score.

1.3 Structure of the Paper

The structure of the paper is as follows. In the next Sect. 2 we briefly describe the questionnaire, sampling methods and sample demographic statistics. Section 3 provides descriptive statistics for the three sets of sentences concerning knowledge, attitudes and behaviours, including rank orderings of the sentences in each set according to their numbers of valid responses, percentages of respondents giving responses that are favourable for sustainable development, mean scores, percentages of ‘don’t know’ and ‘don’t understand’ responses, and simple missing values. In Sect. 4 we describe our three new indexes, namely, the Index of Knowledge of SD, Index of Favourable Attitudes Toward SD and Index of Favourable Behaviours Toward SD. Section 5 reviews correlations among the indexes and results of regressing the Behaviours Index on the other two. The paper ends with a brief summary in Sect. 6.

2 Questionnaire, Methods and Sample Statistics

2.1 Survey Instrument

A three-page questionnaire was developed that contained three sets of sentences which might describe respondents’ knowledge, attitudes and behaviours concerning SD. The complete questionnaire with on-scale means, numbers and percentages of responses is reproduced in Appendix 1. Appendix 2 contains on- and off-scale numbers and percentages including those for ‘don’t know’, ‘don’t understand’ and simply missing values for sentences in Sections A-C identified only by the sentence numbers.

Standardized measures of respondents’ knowledge, attitudes and behaviours were crafted from a subset of the 24 sentences in Section A, 18 in Section B and 17 in Section C, respectively. In each Section, respondents were asked to indicate their level of agreement or disagreement with each sentence on a Likert-type scale running from ‘strongly agree’ (=1), through ‘neutral’ (=3) to ‘strongly disagree’ (=5). For most of the sentences in Section A, ‘strongly agree’ and ‘agree’ responses indicate that the respondent’s knowledge is consistent with themes regarded as necessary and/or favourable for sustainable development. A few sentences were worded negatively as a hedge against erroneous responses, e.g., always checking the same response category resulting from fatigue, boredom, lack of understanding or somewhat playful malice (Sudman and Bradburn 1983). Such sentences are clearly identified for analytic purposes and where necessary, responses are reverse-coded so that percentages indicating views favourable to sustainable development are used correctly. For most of the sentences in Sections B and C, ‘strongly agree’ and ‘agree’ responses indicate that the respondent’s attitudes and behaviours, respectively, are supportive and/or favourable for sustainable development. As with Section A, a few sentences were worded negatively.

While on-scale responses give us information about students’ levels of agreement, disagreement or neutrality regarding sentences in each section, off-scale responses are equally important because they indicate that students did not know how to respond or did not understand the sentence. For any given on-scale response, it is possible that students were not entirely sure how to respond and/or did not fully understand the sentence being considered, but we have no way of measuring levels of such response nuances. However, as we will show shortly, off-scale responses provide useful guides to diverse difficulties with some sentences.

Following these three sections, there was a set of demographic questions concerning gender, age, cultural/ethnic background, postal code and school division.

2.2 Survey Process

Grade 10 students were chosen as the target group for this second phase of work. According to the new learning outcomes developed by the Department of Education for Kindergarten through to Grade 12 students, by Grade 10, students should have received instruction in the basic social, environmental and economic dimensions of sustainability in their social studies, geography and science classes. Further, in Manitoba, Grade 10 is the final year in which students remain together throughout the day in class groups, making them easier to reach for survey purposes.

The questionnaire and appropriate letters of permission received approval from the Brandon University Research Ethics Committee in December 2009. In the first week of January 2010 an email letter was sent from the Sustainable Development Coordinator of the Manitoba Ministry for Education to the Superintendents of all 36 School Divisions in the province. The letter briefly reviewed the history and purposes of our pilot surveys and the new survey, and requested cooperation to ensure successful completion of the initiative. Follow-up phone calls were made to Superintendents who did not respond to the initial letter. Where cooperation was obtained, a representative from our research team contacted schools and teachers, and arranged to have students complete our questionnaire online using SurveyMonkey. Data collection continued to June 2010, and survey summary results were then transformed from an EXCEL format to SPSS for final cleaning and analysis.

After removing cases corrupted in one way or another, there were 1,551 cases in the working dataset, or 10% of the 15,552 students enrolled in Grade 10 in the 2009/10 school year.

2.3 Demographic Characteristics of the Survey Group

Of the 1,439 students who identified their gender, 49.9% were female and 50.1% male (see Appendix 1).

The mean age was 15.6 years, and although the ages ranged from 14 to 32, 86% of respondents were 15 or 16 years of age. Grade 10 students in Manitoba, according to the Department of Education, are usually between 15 and 16 years of age.

Of the 1,384 students who identified their “cultural, ethnic or national background”, 50.9% simply wrote ‘Canadian’, 19.1% wrote ‘European’, 10.1% ‘Asian’, 7.7% Métis and 6.7% ‘First Nations’. Of 36 School Districts, schools in 13 of the districts participated, with 68.9% of respondents coming from the urban school districts of Winnipeg, St. James-Assiniboia, and Brandon, 27.8% from eight rural school districts and 3.3% from northern and remote areas (3.2% from Mystery Lake and 0.1% from Frontier). While the age distribution is representative, and the gender balance probably under-represents females only by a percentage point or so in the age group, it is less clear whether the urban/rural/northern and remote districts are well represented. According to the Enrolment Report September 30, 2009 (Manitoba Education 2010), about 27% of tenth graders are in the Winnipeg, St. James-Assiniboia and Brandon Districts, 10.3% are in the 8 rural Districts in our sample and 4.8% in our two northern and remote areas. However, when considering the urban/rural/northern and remote distribution of Grade 10 students across all school districts, the representativeness of the survey sample is closer. Enrollment in the 9 school districts (including a portion of Division scolaire Franco-Manitobaine) that serve the 4 largest urban centres in the southern part of the province (Winnipeg, Brandon, Portage La Prairie and Steinbach) accounts for 64.7% of all Grade 10 students; enrollment in the rural districts account for 30.5% of enrollment, and enrollment in the northern and remote districts of Mystery Lake and Frontier accounts for the remaining 4.8%.

So, our sample should not be regarded as fully representative of tenth graders in School Districts across the whole province, although it is close. Although it would be possible to add weights to responses to obtain sample sizes more representative of the Manitoba tenth grade school population, we have not adopted that strategy because completion of the questionnaire was entirely voluntary and we suspect that the students who took the time to complete our questionnaire are probably not like those who did not bother with regard to issues surrounding sustainable development. Apart from this suspicion, we know that we did not have participation from all school districts or all schools in some districts. So, many students did not have the opportunity to opt in or out.

3 Descriptive Statistics

In this section we provide two sets of observations: the first on the responses themselves and what they might tell us about the levels of student knowledge, attitudes and behaviours (the baseline) and the second on what the responses mean for the construction of the standardized measures. There is much more information in the tables than we could conveniently summarize in our overview. However, the aim of the overview is not to end a conversation but to begin one. As different people reflect on the results reported here, we hope that they will find ways to improve our understanding of the strengths and limitations of the specific sentences used in each section. With more work we should be able to improve our understanding of what sentences best describe the knowledge, attitudes and behaviours concerning SD that we hope to nourish in our students today and adult citizens tomorrow.

Because rank order scores are ordinal numbers, strictly speaking the mathematical functions of addition and so on are not applicable. So one might prefer to rank order the support indicated for each sentence by the percentage of respondents strongly agreeing (or disagreeing for negatively worded sentences) or perhaps the total percentage of favourable responses (i.e., percent strongly agreeing plus percent agreeing, or disagreeing for negatively worded sentences). However, because quite a bit of information is lost by neglecting the responses that are unfavourable or neutral, one might prefer to treat the rank order numbers as if they were cardinal. This has the particular advantage of allowing one to use mean or average scores as summary measures of favourable or unfavourable responses. Given these options, for each sentence, we have provided both percentages of respondents strongly agreeing (or disagreeing in cases of reverse-coding) and mean scores of all responses. Mean scores have been calculated as the average of all the on-scale scores divided by the number of respondents responding on scale, with 1 being highest and 5 being lowest on the scale. Our own preference is for mean scores.

To simplify our discussion, hereafter expressions like ‘Section A Sentence #5’ will be abbreviated to ‘A.S5’.

3.1 Section A: Knowledge

3.1.1 Implications for the baseline

Table 1 below presents a summary of the responses to the first section of the survey instrument, that presented 24 statements describing aspects of sustainability that one might expect a Grade 10 student to know. The table lists the on-scale responses (the Likert scale of 1–5 from strongly agree to strongly disagree).

Table 1 Summary, Section A: Ranking with valid entries (on scale responses of 1–5), ordered by Mean score on knowledge sentences
Full size table

Reading across the first row, one finds that the overall average score of 1.71 (out of 5) for A.S3 (Protecting the environment is necessary for SD) gives it a first place rank among mean scores. 46.7% of respondents strongly agreed with A.S3 which ranks it first place in terms of the percentage of ‘strongly agreed’ (favourable) responses for sustainable development. While the top 6 sentences are ranked the same for both mean and percentage of entries that strongly agree (i.e., A.S3, 12, 5, 17, 13, 16 are ranked first, second,…sixth by both measures), this equivalency only happens 3 more times in the set of 24 sentences in Section A.

Interestingly, two reverse-coded sentences are ranked twenty-third and twenty-fourth by both measures in this section, namely, A.S*24 (SD allows the environment to be radically altered in order to create/maintain jobs.) and A.S*8 (SD is not dependent on gender equality.)

The sentences were crafted to reflect a range of themes on the environmental, social and economic dimensions of sustainability. It is interesting to note that the top five ranked responses were on statements directly related to the environmental dimension. The next six (those statements receiving above 70% strongly agree and agree responses) reflected a mix of social, economic and environmental themes. While we were pleased to see this signal that students might be grasping the multidimensional aspects of sustainability, we also noted that a full third of the respondents did not respond as positively to statements suggesting that knowledge of global economic equity issues is important (A.S15—SD results in fair distribution of goods and services to all people around the world; A.S9—the elimination of poverty is necessary for SD; and A.S18 on the Millennium Development Goals). On reflection, A.S15 seemed redundant given A.S8-A.S10, A.S18 and A.S20. So it was dropped from the composite index.

3.1.2 Implications for the Composite Index

Table 2 reveals key issues with several sentences when the off-scale responses (don’t know, don’t understand or didn’t answer) are also considered.

Table 2 Section A, Rankings including off scale responses
Full size table

Examining Table 2, the sentence ranked first on both measures of confusion about the sentence meaning was A.S18 (SD requires achieving the United Nations’ Millennium Development Goals.), with 20.2% ‘don’t know’ and 16.8% ‘don’t understand’ responses, respectively. The Millennium Development Goals (MDGs) are a central international building block for global sustainability, and figure prominently in the long term planning of UN agencies (including UNESCO) as well as other multilateral and bilateral development assistance agencies. While the MDGs may not be covered in the Manitoba Education curriculum, resources for teaching the MDGs are being developed in many other jurisdictions. Granting that the MDGs will not be achieved by the end of the Decade of ESD (2015), their achievement is nevertheless an important theme of ESD and SD. So A.S18 was not expendable from the composite index.

A.S21 (Cell phone production and usage does not consume a lot of resources.) and A.S23 (I have taken a course in which sustainable development was discussed.) were ranked second and third, respectively, by the ‘don’t know’ response. While neither sentence directly expresses a theme of ESD or SD, the fact that 11.7% of respondents did not seem to know if they had taken a course in which SD was discussed in any way, for any length of time, is a bit worrying. The cell phone sentence may have suffered for lacking a clear basis of comparison (e.g., a lot of resources compared to cars, television sets or even Big Macs?) or maybe for students lacking basic information about the production of cell phones themselves. As a test of “applied knowledge” (compared to themes or concepts described in the majority of sentences) it was not particularly successful. Both have been excluded from the composite index.

One finds that for A.S*24 (SD allows the environment to be radically altered in order to create/maintain jobs), 11.0% of respondents did not know what to say (for whatever reason) and 9.4% did not understand the sentence, while for A.S*8 (SD is not dependent on gender equality), 6.6% did not know what to say and 7.2% did not understand the sentence. Since both responses (‘don’t know’ and ‘don’t understand’) indicate some confusion about the meaning of the sentences, both sentences were candidates for exclusion. Because we had no other sentence referring specifically to gender equality and this is a theme of central importance to ESD and SD as understood by UNESCO, we kept A.S*8 in our composite index but discarded A.S*24. Because adult and student respondents had trouble with our gender equality sentences in our pilot studies too, we believe the problems with A.S*8 are not simply the result of its confusing sentence structure. Gender parity has notoriously been very slowly introduced into UNESCO policies and programs over many years, given the diversity of opinion in many parts of the world. So it would perhaps be remarkable if there were no confusion in the minds of Manitoba tenth grade students.

3.2 Section B: Attitudes

3.2.1 Implications for the Baseline

Table 3 presents a summary of the responses to the second section of the survey instrument, that presented 18 statements describing attitudes towards sustainability that one might expect a Grade 10 student to hold. The table lists the on-scale responses (the Likert scale of 1–5 from strongly agree to strongly disagree).

Table 3 Summary, Section B: ranking with valid entries (on scale responses of 1–5), ordered by Mean score on attitudes sentences
Full size table

The first row of Table 3 tells us that B.S14 (Males and females should have equal access to all kinds of education and employment.) had the largest number of valid responses (1,441), the largest percentage of ‘strongly agree’ responses (56.9%) and the lowest mean score (1.67). As well, the last row of Table 4 below tells us that this sentence had the smallest percentage of ‘don’t know’ responses (2.1%). After the confusion over the gender equality sentence in the knowledge section (A.S*8), these responses to B.S14 are very encouraging. Most of our sampled students apparently appreciate the importance of boys and girls, men and women having equal access to two of the greatest contributors to a sustainable good life, i.e., a good education and a good job.

Table 4 Section B rankings including off scale responses
Full size table

The second row of Table 3 also has an encouraging message. B.S1 (Every person should receive education that teaches the knowledge, values and skills necessary for sustainable living in a community.) ranked second for number of valid responses (1,440), percentage of ‘strongly agree’ responses (39.8%) and mean score (1.80). The second last line of Table 4 tells us that B.S1 also had the second smallest percentage of ‘don’t know’ (2.3%) and ‘don’t understand’ (2.4%) responses. Reading down the ‘strongly agree/disagree’ and ‘mean’ score columns, one finds that of the 18 rows in Section B Part 1, only the first two rows have complete agreement on the rank order of sentences.

Unlike the knowledge section in which the environmental dimension dominates the top five ranked responses, the top five ranked responses in the attitudes section reflect a mix of positive social and environmental attitudes. It was encouraging to see that an economic equity issue (B.S6—it is important to find ways to reduce poverty) also made it into the top five for positive attitudes. However, over a third of respondents did not respond as positively to another sentence dealing with global equity (B.S7—SD will not be possible until wealthier nations stop exploiting workers in poorer nations).

3.2.2 Implications for the Composite Index

Table 4 reveals key issues with several sentences when the off-scale responses (don’t know, don’t understand or didn’t answer) are also considered.

The first two rows of Table 4 rank B.S15 (It is useful to estimate the dollar value of the services that the ecosystem provides to us.) in first place and B.S7 (SD will not be possible until wealthier nations stop exploiting workers in poorer nations.) in second place for the ‘don’t know’ and ‘don’t understand’ responses. Both statements may require additional knowledge and experience not yet available to Grade 10 students; further, while both sentences may be true and agreement with them may be favourable for SD, neither is clearly articulated in UNESCO literature, making both possibly expendable for our composite index. Because B.S15 was also ranked last in the ‘strongly agree’ column (13.0%) and second last in the ‘mean’ score column, it was dropped from the composite. Because of the importance of B.S7 as a statement on global equity, it has been retained. B.S13 was dropped on further reflection because it seemed to require guessing what the phrase “economy is doing poorly” would imply with respect to job creation. B.S18 has also been dropped from the composite as the wording closely duplicates B.S17 and the mean scores and strongly agree percent responses are comparable.

3.3 Section C: Behaviours

3.3.1 Implications for the Baseline

Table 5 presents a summary of the responses to the third section of the survey instrument that presented 17 statements describing behaviours supportive of sustainability that one might expect a Grade 10 student to demonstrate. The table lists the on-scale responses (the Likert scale of 1–5 from strongly agree to strongly disagree).

Table 5 Summary, Section C: ranking with valid entries (on scale responses of 1–5), ordered by Mean score on behaviour sentences
Full size table

C.S17 (I give men and women, boys and girls the same level of respect.) had the lowest mean score (2.03) while C.S*9 (I treat people respectfully, except those who have racial backgrounds different from my own.) had the largest percentage of respondents expressing disagreement (44.1%). C.S17 has the second largest percentage of respondents strongly agreeing (38.4%) and C.S*9 has the second lowest mean score (2.26). C.S17 also has the advantage of having the smallest percentage of ‘don’t know’ responses, with C.S*9 ranked next at 1.9%, Both sentences tap into respondents’ egalitarian inclinations, one concerning race and the other concerning gender. While the numbers are relatively small, C.S*9 has a larger percentage than C.S17 of ‘don’t understand’ responses, (4.2%) compared to (2.4%).

We note that there is only one row out of the 17 in Section C Part 1 with a sentence ranked the same in the ‘strongly agree/disagree’ and ‘mean’ score columns. C.S1 (I choose to walk or bike to places instead of using a motor vehicle.) ranks fifth with 17.5% strongly agreeing and a mean score of 2.66.

With no sentence achieving a mean score below 2, however, it would appear that the practice of positive behaviours is not as strong as the indications of knowledge of sustainable development themes and attitudes supportive of sustainable development.

3.3.2 Implications for the Composite Index

Table 6 presents the off-scale responses (don’t know, don’t understand or didn’t answer) for the behaviours section.

Table 6 Section C rankings including off scale responses
Full size table

The first row of Table 6 ranks C.S11 (I try to avoid buying goods from companies with poor track records on caring for their workers or the environment.) in first place for the ‘don’t know’ and ‘don’t understand’ responses. However, because C.S11 had a relatively strong correlation with the composite (r = .52), it was retained in the composite. After trying several combinations, C.S4, C.S5 and C.S*9 were excluded from the composite index in the interest of achieving an index Cronbach alpha score ≥0.80. (More on Cronbach scores in next section.)

4 Indexes of Knowledge, Attitudes and Behaviours

Table 7 lists indexes of knowledge of, and favourable attitudes and behaviours toward sustainable development, with numbers of respondent cases used in the calculations, Cronbach alpha reliability coefficients, scale means and item-total correlations. Each of the indexes was built on a subset of the sentences in Sections A-C, using the information in Tables 1–6 plus the guidelines drawn from the UNESCO documents cited in the introduction and the results of our pilot studies.

Table 7 Indexes of knowledge of, favourable attitudes and behaviours toward sustainable development, with numbers of respondents, Cronbach Alphas, scale means and item-total correlations
Full size table

The first two columns of Table 7 tell us that for the 20-item Index of Knowledge of SD, there were 673 valid cases used in the calculation that produced a Cronbach reliability coefficient of α = 0.89. Generally speaking, alpha coefficients above 0.70 are regarded as acceptable and those above 0.80 as relatively high. Alpha coefficients of indexes are important because their square roots indicate upper bounds for correlations with any other measures, e.g., the maximum correlation of an index with an α = 0.80 would be r = .89 (Carmines and Zeller 1979). Alpha coefficients measure the inter-correlations among the sentence items in a set based on pairwise correlations, and more precisely they measure “the proportion of variance attributable to common sources” (Viswanathan 2005, p. 27). As explained in Michalos et al. (2011), following Fayers and Hand (2002) and earlier researchers like Bollen (1984), our approach to index construction is more in the clinimetric than the psychometric tradition. For present purposes, the main difference is that “we are defining the thing being measured in terms of the variables that we select to measure it. In contrast with the psychometric approach, we are not postulating that something exists but are merely constructing an index which is convenient for some purpose” (Fayers and Hand 2002, p. 237). With regard to composite indexes aimed at capturing the central features of UNESCO’s socially constructed, broadly understood umbrella concepts of sustainable development and education for sustainable development, the short quotation from Viswanathan hits the nail right on the head with its reference to “common sources”. There are many overlapping sources of the features of these concepts, some of which are fairly highly correlated and some not. Accordingly, our procedures for dropping and adding sentence items to the indexes had to be pragmatic, attending to both conceptual and numerical information.

This 20-item index had a scale mean of 2.12 (out of 5) and a standard deviation of 0.37. Its average item-total correlation was r = 0.51, with a range from r = −0.217 to r = 0.650. If the sentence concerning gender equality (A.S*8) were dropped from the index, the average correlation would increase to r = 0.55, but some important content validity would have been lost. The main message we take home about A.S*8 is to replace it with a positive sentence in any future studies.

Comparing results here with those of the 16-item knowledge index in our student pilot study, one finds considerable improvement. The knowledge index in the pilot study had an α = 0.79 and an average item-total correlation of r = 0.39. While alpha coefficients can be increased by increasing the numbers of sentences in the relevant set while holding all other features constant, the fact that our average item-total correlation was also higher in this study than in the pilot shows that we have managed to reduce the random error in our index scores.

The second two columns of Table 7 tell us that for the 15-item Index of Attitudes Favourable Toward SD, there were 992 valid cases used in the calculation that produced a Cronbach reliability coefficient of α = 0.84. This 15-item index had a scale mean of 2.14 and a standard deviation of 0.26. Its average item-total correlation was r = 0.46, with a range from r = 0.379 to r = 0.569. Comparing these results with those of the 11-item attitude index in our student pilot study, one finds improvement again. The attitude index in the pilot study had an α = 0.77 and an average item-total correlation of r = 0.41.

Finally, the last two columns of Table 7 tell us that for the 14-item Index of Behaviours Favourable Toward SD, there were 968 valid cases used in the calculation that produced a Cronbach reliability coefficient of α = 0.83. This 14-item index had a scale mean of 2.83 and a standard deviation of 0.37. Its average item-total correlation was r = 0.46, with a range from r = 0.018 to r = 0.659. If C.S*10 (on composting) were dropped from the set, the average item-correlation would increase to r = 0.50. Because composting is a relatively easily understandable and individually doable sort of behaviour favourable for sustainable development, we did not drop the item. Comparing these results with those of the 14-item behaviours index in our student pilot study, one finds improvement again. The attitude index in the pilot study had a relatively low α = 0.63 and a low average item-total correlation of r = 0.26.

In our pilot study, we noted that the indexes were not very useful for distinguishing diverse levels of knowledge, attitudes and behaviours because such a high percentage of respondents gave relatively correct or favourable responses. Our revised indexes performed much better. In large classes, one way to identify superior performance on an exam is to look at the scores that are two standard deviations from the mean. With a bell-shaped normal distribution, one gets about 2.3% of respondents scoring two standard deviations above the mean. For our knowledge and attitudes indexes, we had 2.6% scoring that high, and for our behaviours index we had 3% scoring that high. So our distributions here are much more nearly normal and therefore more useful than any we had in our pilot study.

4.1 Implications for the Baseline

In summary, using the composite index, we would suggest that Grade 10 students in Manitoba in 2010 achieved the following on a scale of 1–5, with 1 being the highest:

  • On knowledge of central sustainable development themes and concepts: 2.12

  • On attitudes favourable to sustainable development: 2.14

  • On behaviours supportive of sustainable development: 2.83

Bearing in mind the caveats throughout this paper with respect to representativeness of this sample, with respect to the wording of several sentences that have been included in the composite, and with respect to the problem that the identification of specific behavioural changes required by individuals (especially 15 year olds) in the interest of sustainable development are not easily found, we believe that this baseline nevertheless will be useful as a starting point against which to assess progress in the coming years.

5 Bi-Variate and Multi-Variate Relations

Table 8 lists the Pearson Product-Moment Correlations among our three indexes, based on data from 1,442 respondents. The Knowledge and Attitudes Indexes have a relatively high association (r = 0.62) compared to the associations between the Knowledge and Behaviours Indexes (r = 0.39) and Attitudes and Behaviours Indexes (r = 0.44). These correlations are all higher than their counterparts in the student pilot study and all highly significant at p < .0001.

Table 8 Correlations among indexes for knowledge of, favourable attitudes and behaviours toward sustainable development, N = 1,442, p < .0001
Full size table

Table 9 summarizes the results of regressing the Behaviour Index on the other two Indexes, mindful of our need to make individual “sustainable behaviours become daily habits” and our assumption that knowledge and favourable attitudes should be our main instruments of transformation. If knowledge and favourable attitudes were sufficient to explain respondents’ behaviours, then the second column and row entry of Table 9 would have 100% instead of the 21% found there. Based on a sample of 1,424 respondents, our assumption leaves about 80% of the variation in Behavioural Index scores unexplained. The Beta (β) values are standardized regression coefficients, with means of zero and standard deviations of one. Figuratively speaking, they may be interpreted as percentages of a full step, i.e., when the values of all but one predictor in an equation are held constant and the value of the unbound predictor moves one full step, the value of the dependent variable will move a certain percentage of a step. Thus, the last row of the table tells us that if the Favourable Attitudes Index score increased by one full step (one standard deviation unit), then the Favourable Behaviours Index score would increase by 32% of a step. Comparing the Beta value entries in the last and second last rows, we see that attitudes are more influential than knowledge, since an increase in the Knowledge Index score of one full step would only increase the Favourable Behaviours Index score by 20% of a step. To some extent the correlations in Table 8 gave us a clue to what we would find in Table 9, but the regression analysis has the added benefit of revealing the relative influence of each predictor in the presence of the other. In the student pilot study, the Knowledge and Favourable Attitudes Indexes could only explain 14% of Favourable Behaviours scores, with attitudes again more influential than knowledge.

Table 9 Regressions of favourable behaviours index on knowledge and favourable attitudes indexes
Full size table

6 Conclusion

The aim of this paper was to develop standardized measures of tenth grade students’ knowledge, attitudes and behaviours concerning sustainable development as those concepts are understood in the United Nations Educational, Scientific and Cultural Organization (UNESCO Education Sector 2006) and to test the hypothesis that knowledge and favourable attitudes toward SD would lead to favourable behaviours. The survey questionnaire used in this study was a refined version of the student questionnaire used in a pilot study 2 years earlier. While further work on the questionnaires and indexes is still necessary, some clear and important progress was made over our earlier study. We now have much better information about all the sentence items used in the indexes and each index has better measurement properties than anything we had in the pilot study. The fact that only 21% of the variation in our Favourable Behaviours Index scores could be accounted for by our Knowledge Index and Favourable Attitudes Index scores implies that our indexes or our basic hypothesis about their relationships or both may be defective. Although our sample was close but not fully representative of the total tenth grade population in all the Manitoba school districts, it was large enough that we suspect that the relationships identified here are fairly consistent with what we would find in a fully representative sample. Needless to say, we would like to have additional opportunities to refine our measures, diversify our samples and increase our explanatory variables in order to account for the roughly 80% of unexplained variance in individual behaviour.