Introduction

The impact of social and cultural factors on cognitive development has long been recognised. Piaget himself (1977) acknowledged the impact of social factors and peer interaction on cognitive development, and more recent studies have generally confirmed this view (Downing et al. 2007). However, relatively few definitive studies evaluate the impact of PBL approaches outside the field of medicine, particularly the impact of problem-based approaches on the development of metacognition in first year university students, and recent research tends to concentrate on the impact of ‘Learning to Learn’ type courses upon the development of undergraduates (Kwan 1999). In those studies undertaken outside medicine (e.g. Kwan and Ko 1999; Dean 1999) a variety of different measures have been used to assess the impact of PBL approaches, and these have not always focused upon the development of metacognition.

Using the Learning and Study Strategies Inventory (LASSI) as a measure of student perceptions of their thinking, or metacognition, this study samples two groups of first year undergraduates in Hong Kong and identifies significant differences in metacognition between students taking first year building and construction courses, which have adopted a problem-based approach to learning, and those taking similar building and construction courses in a non-PBL environment.

Metacognition

Perhaps the simplest definition of metacognition is that it is ‘thinking about thinking’ (Bogdan 2000; Flavell 1999; Metcalfe 2000), but metacognition also involves knowing how to reflect and analyse thought, how to draw conclusions from that analysis, and how to put what has been learned into practice. In order to effectively solve problems, students often need to understand how their mind functions. In other words, they need to perceive how they perform important cognitive tasks such as remembering, learning and problem solving. Therefore, problem-based learning should produce significant metacognitive development in undergraduates when compared to non problem-based approaches which do not always require the same reflective performance.

Kluwe (1987) refined the concept of metacognition by noting two characteristics: the thinker knows something about his or her own and others’ thought processes, and the thinker can pay attention to and change his or her thinking. This latter type of metacognition Kluwe calls ‘executive processes’. Hacker (1998) points out the difference between cognitive tasks (remembering things learned earlier that might help with the current task or problem) and metacognitive tasks (monitoring and directing the process of problem solving), stressing the importance of learning more about thinking. Cornoldi (1998) emphasises the role of learners’ beliefs about thinking and makes the point that if students feel confident that they can solve problems, they tend to do better work. In defining metacognition as ‘thinking about thinking’ or ‘second-order cognition’, Weinert (1987) acknowledges that purpose, conscious understanding, ability to talk or write about tasks, and generalisability to other tasks are also important factors in determining whether a given task is metacognitive. This viewpoint is supported by Brown (1987) who agrees that metacognition requires the thinker to use and describe the process of mental activity. Many other researchers also make the point that metacognition is best defined by acknowledging that it is both knowledge about, and control over thinking processes (Allen and Armour-Thomas 1993). Vadhan and Stander (1994) clearly distinguish between ordinary thinking and awareness and understanding of thinking, and this is a theme elaborated by Hacker (1998) who divides metacognition into three types of thinking:

  • Metacognitive knowledge: What one knows about knowledge.

  • Metacognitive skill: What one is currently doing.

  • Metacognitive experience: One’s current cognitive or affective state.

Therefore, whilst cognition focuses on solving the problem, metacognition focuses on the process of problem solving (Marchant 1989).

Metacognition and PBL

In addition to the knowledge people have about how they use their thoughts and strategies (Brown 1987), knowledge about how much they will be able to learn, and what kinds of strategies they use (Gleitman 1985; Weinert and Kluwe 1987); people also have a set of general heuristics. For example, how they plan, set goals, and process feedback (Frese et al. 1987). The assumption is that these general heuristics can be either conscious or automatic (Brown 1987; Flavell 1987) and they may be highly generalised or specific. Biggs (1999) identifies the aim of undergraduate education as getting students to develop the functioning knowledge which allows them to integrate the academic knowledge base (declarative knowledge), skills required for that profession (procedural knowledge) and the context for using them to solve problems (conditional knowledge). Hmelo et al. (1997) argue that problem-based learning by its very nature requires a different way of using knowledge to solve problems, and it is this ‘functioning’ knowledge that involves the metacognitive processes identified above. Consequently, because problem-based learning uses real world cases or problems as the starting point, the processes involved in solving these problems should lead to the development of the two characteristics of metacognition defined by Kluwe (1987). Although there are many varieties of problem-based learning, they all require the successful student to monitor and direct the process of problem solving, bringing memory of concepts and processes learned earlier to bear upon the current problem. In fact, the general sequence of problem-based learning: the motivational context of learning is set up by a real-life problem; learners are activated through group, peer and facilitator interaction; a knowledge base of relevant materials is constructed and applied to deal with the case; and the case is then reviewed, requires reflection upon declarative, procedural, and conditional knowledge. Therefore, problem-based learning should, in theory at least, be ideally tailored to the rapid development of metacognition in undergraduates.

Metacognition and the Learning and Study Strategies Inventory (LASSI)

Metacognition can be assessed in a number of ways but one of the most popular methods currently in widespread use is through the use of questionnaires which require students’ to report their perceptions about their thinking and problem-solving skills and strategies. It is generally accepted that most students who struggle at university could improve their performance considerably if they understood the learning process better. Weinstein and Palmer (1988) assert that learning is more effective when we engage in thinking about the process of learning, thinking, and problem-solving. As a result of her work in the field of strategic learning at the University of Texas at Austin, Weinstein (1987) developed the Learning and Study Strategies Inventory (LASSI) which is now the most widely used learning inventory in the world. The LASSI measures student’s perceptions of their study and learning strategies and methods. In other words, it is a measure of the students thinking about their thinking or metacognition. The second version (Weinstein and Palmer 2002) of the tool consists of ten scales, and eighty items which provide an assessment of students’ awareness about and use of learning and study strategies related to skill, will and self-regulation components of strategic learning. Research has repeatedly demonstrated that these factors contribute significantly to successful study, and that they can be learned or enhanced through educational interventions such as learning and study skills courses (Weinstein 1994a, b; King 1991; Letteri 1992; Hanley 1995). The LASSI provides standardised scores for the ten different scales and provides students with a diagnosis of their strengths and weaknesses, compared to other students, in the areas covered. It measures three main areas of ‘strategic learning’:

Skill component of strategic learning

These scales examine students’ perception (metacognition) of their learning strategies, skills and the thought processes related to identifying, acquiring and constructing meaning for important new information, ideas and procedures. The LASSI scales related to the skill component of strategic learning are:

  • Information processing—the ability to process ideas by mentally elaborating on them and organising them in meaningful ways.

  • Selecting main ideas—the student’s ability to identify the important information in a learning situation.

  • Test strategies—the student’s ability to prepare effectively for an examination and to reason through a question when answering it.

The will component of strategic learning

These scales measure students’ perceptions of their receptivity to learning new information, their attitudes and interest in college, their diligence, self-discipline, and willingness to exert the effort necessary to successfully complete academic requirements, and the degree to which they worry about their academic performance. The LASSI Scales related to the will component of strategic learning are:

  • Attitude—the student’s perceived motivation and interest to succeed in their study, and willingness to perform the tasks necessary for academic success.

  • Motivation—the extent to which the student accepts responsibility for performing those tasks by using self-discipline and hard work.

  • Anxiety—the degree of anxiety perceived by the student when approaching academic tasks.

The self-regulation component of strategic learning

These scales measure how students’ perceptions of how they manage, or self-regulate and control, the whole learning process through using their time effectively, focusing their attention and maintaining their concentration over time, checking to see if they have met the learning demands for a class, an assignment or a test, and using study supports such as review sessions, tutors or special features of a textbook. The LASSI Scales related to the self-regulation component of strategic learning are:

  • Concentration—the student’s perceived ability to focus his or her attention, and avoid distractions, while working on school-related tasks like studying.

  • Time management—the student’s perception of the extent to which they create and use schedules to manage their responsibilities effectively.

  • Self-testing—the student’s awareness of the importance of self-testing and reviewing when learning material, and use of those practices.

  • Study aids—the student’s perceived ability to use or develop study aids that assist with the learning process.

Method

Design

Starting from 2005–2006 the LASSI was offered to all first-year undergraduate students at City University of Hong Kong in order to help them monitor and develop appropriate learning attitudes and strategies, and maximize the opportunity for students to enjoy a successful learning experience during university and beyond. Group A consists of undergraduate students on a degree programme in building who completed both entry and interim LASSI questionnaires in September 2005 and January 2007 respectively (N = 33). This group was matched for age, gender and housing type with a sample from another building programme, composed of associate degree students (Group B), who completed LASSI at the same times as part of the institutional LASSI initiative. Matching for these particular variables was considered necessary in the light of previous published research conducted in Hong Kong which clearly suggested that age, gender, and housing type were significant variables in terms of LASSI score (Downing et al. 2007). The undergraduates following the full degree programme in the first programme follow a course of study and assessment in their first year which is distinctly non problem-based learning (non-PBL) whereas the associate degree student group from the second programme follow an exclusively problem-based approach. This was therefore an opportunity to compare the impact of PBL across two programmes in the same discipline. This is therefore a related samples design using matched pairs and data is analysed using the Wilcoxon signed ranks test for related samples because the data from LASSI is ordinal (Plutchik 1974). Therefore, the Wilcoxon test was chosen to ensure good inferential rigour.

Materials

The measure of Metacognition used for both entry and final tests is the Learning and Study Strategies Inventory (Second Edition) which is commercially available (Weinstein and Palmer 2002).

Participants

First year undergraduates from the building discipline at a Hong Kong university (N = 66). Group A (n = 33) were selected from full-time degree programme students embarking on their first year of study as undergraduates. Group B (n = 33) were selected from full-time associate degree students also embarking on their first year of study. The participants were matched for age (the range for both groups was 20–25 years), gender (Table 1) and housing type. Housing type has proved a significant factor in academic performance in Hong Kong (Downing et al. 2007) so we included this in our matching criteria.

Table 1 Gender distribution/matching
Full size table

‘A’ level data for each group was also gathered because the entry requirements for each group of students are different given their admission to degree and associate degree programmes. ‘A’ level (AL) scores are calculated on a ‘point’ basis in Hong Kong with 2 AL subjects or 1 AL subject plus 2 Advanced-supplementary (AS) level subjects being counted. For AL subjects, grade A = 10 points, B = 8 points, C = 6 points, D = 4 points, E = 2 points; whilst for AS level subjects, grade A = 5 points, B = 4 points, C = 3 points, D = 2 points, E = 1 points. Thus, the maximum score for each student should not exceed 20 points. Table 2 shows the ‘A’ level scores for each group on entry to their respective programmes. The non-PBL degree programme students not surprisingly score significantly higher than their associate degree programme counterparts in terms of ‘A’ level achievement on entry.

Table 2 ‘A’ level points for each group on entry
Full size table

Learning contexts

The problem-based curriculum involved students working in small tutorial groups of between five and six trying to understand, explain and solve problems set by the tutor. The commonly used ‘seven-jump’ approach (Vermunt 2007) was adopted in which PBL is systematically structured into seven broad steps: (1) clarifying terms and concepts not readily understood; (2) defining the problem; (3) analysing the problem; (4) summarising the various explanations of the problem into a coherent model; (5) formulating learning objectives; (6) individual study activities outside the group and; (7) report and synthesise the newly acquired information. Students have approximately 3 h per week of tutor contact time at their disposal over a 13 week semester but this is organised flexibly according to student need. Therefore, some groups might wish to meet for longer tutorials early in the semester whereas others chose to meet for shorter periods on a weekly or twice weekly basis. In the early tutor groups students analyse the problem(s) and formulate learning outcomes, questions to which they should find answers through individual study. When the individual study is complete, the tutor group meets again to present what they have learned about the problem(s). At this stage, the tutor clarifies and analyses the acquired information and ideas with the intention of assisting students to integrate their knowledge. Throughout the semester the tutor’s main task is one of facilitating learning and providing timely and appropriate scaffolding when required by the tutorial group.

In the non-PBL group a more traditional approach is taken whereby the subject matter is determined largely by the lecturer and the focus is on declarative and procedural knowledge, rather than functioning problem-based knowledge (Leinhardt et al. 1995; Biggs 1999). Contact time is Again 3 h per week over a 13 week period but this is organised into a 2 h lecture and 1 h tutorial per week for each of the 13 weeks. In lectures, students are presented with clarifications and explanations of the subject matter, and weekly tutorials are used to deepen understanding of the acquired knowledge, clarify individual problems and provide feedback on formative and summative assessments. Subject matter, learning outcomes assessment and feedback remain largely in the hands of the lecturer or tutor although students are free to choose their own learning strategies and approaches.

Results

Entry scores for the two groups

The results for the LASSI entry scores for Groups A (non-PBL degree students), and B, (PBL associate degree students) demonstrate that the degree entrants with higher ‘A’ level scores also score significantly higher on the LASSI than their associate degree counterparts, and that this statistically significant difference is evident across all ten items of the LASSI questionnaire. In fact the overall means demonstrate a difference of about ten percentiles between the two groups (see Figs. 1 and 2).

Fig. 1
figure 1

LASSI Entry Scores (three components and overall) of Group A (non-PBL) and Group B (PBL) students

Full size image
Fig. 2
figure 2

LASSI Entry Scores (10 scales) of Group A (non-PBL) and Group B (PBL) students

Full size image

Final scores for the two groups

The mean final scores, taken after 15 months and three semesters of study in the different curriculum environments (shown in Figs. 3 and 4) demonstrate a dramatic reversal of this situation with the associate degree students, who experienced the PBL curriculum, scoring significantly higher on all ten items of the LASSI. Overall they had added 18 percentiles to their entry score and overtaken the non-PBL group.

Fig. 3
figure 3

LASSI Final Scores (three components and overall) of Group A (non-PBL) and Group B (PBL) students

Full size image
Fig. 4
figure 4

LASSI Final Scores (10 scales) of Group A (non-PBL) and Group B (PBL) students

Full size image

The mean difference between the entry and final scores on the LASSI for both groups is shown in Figs. 5 and 6 and demonstrates the significant differences over the 15 month period between the non-PBL and PBL students. The results in Figs. 1–6 were all statistically significant at P < .01 or below.

Fig. 5
figure 5

Mean difference LASSI scores (3 components and overall) of Group A (non-PBL) and Group B (PBL) students

Full size image
Fig. 6
figure 6

Mean difference LASSI scores (10 scales) of Group A (non-PBL) and Group B (PBL) students

Full size image

Discussion

We hypothesised that, despite the differences in ‘A’ level entry scores, the students who followed the PBL curriculum (Group B) should show greater improvement in their LASSI scores over the 15 month period of the natural experiment. However, the extent and significance of this difference surprised even the researchers on this project. Whilst significant differences in scores on academic tests (Webster and Riggs 2006) and the LASSI (Kwan and Ko 1999) have been shown before, the former have usually been within the field of medicine and the latter immediately subsequent to ‘learning to learn’ type interventions. In this case, the events occurred naturally and we were able to take advantage of these conditions to conduct a natural experiment. In fact, quite separately from this study, the programme involved in teaching Group A had already decided to adopt a problem-based approach in the future as part of its curriculum development programme.

The ‘will’ component

The LASSI can be broken down into scores for three main components and ten items as identified above, and the greatest difference observed from these results involves the ‘will’ component which has often proved the most resistant to change (Hoban et al. 2001; Holzer 2002; McCall 2002). This component consists of three items from the questionnaire; motivation, attitude and anxiety. Anxiety is a reversed scale meaning that the higher the percentile scores the lower the anxiety levels. Both groups showed improvement on this item but with the PBL group showing a greater reduction in anxiety levels than their non-PBL counterparts (Fig. 6). The situation for both motivation and attitude scales is somewhat different in that the PBL group (B) showed a significant increase in their levels of motivation whereas the group A showed a reduction in motivation over the first 15 months of undergraduate study. The operational definition of the motivation scale for the LASSI questionnaire suggests that it assesses students’ diligence, self-discipline, and willingness to exert the effort necessary to successfully complete academic requirements. Therefore, students who score low on this scale need to accept more responsibility for their academic outcomes and learn how to set and use goals to help accomplish specific tasks. Clearly the use of a problem-based approach has facilitated the development of more confidence (less anxiety) and has substantially enhanced student levels of motivation. This is probably because knowing something about one’s own thought processes and recognizing that it is possible to change one’s thinking (Kluwe 1987) are intrinsically motivating, giving the student a greater sense of control over, and satisfaction with what they produce (Collins and Amabile 1999). A similar pattern is observable with the attitude scale where Group B shows a significant increase and Group A shows a decrease over the same timescale. The attitude scale assesses students’ attitudes and interest in university academic success, examining how facilitative or debilitating their approach to university and academia is for helping them to get their work done and be successful. Students who score low on this scale may not believe university is relevant or important to them and may need to develop a better understanding of how university and their academic performance relates to their future life goals. The improvement in scores for the problem-based group might be partly the result of the use of real-world examples as problems to be considered. In other words, the phenomenon of situated cognition (Hung 2002) where the more true to life the task is, the more meaningful the learning will be. The scaffolding provided by the teachers involved in a PBL approach should also leave students feeling supported by and more positive towards academic pursuits.

The ‘skill’ component

Both groups showed improvement in terms of their ability to select main ideas (SMI) or identify the important information in a learning situation, and their ability to prepare effectively for an examination and reason through a question when answering it (TST). Once again the improvement in the PBL group was significantly greater than in the non PBL group suggesting considerable development of what Biggs (1999) calls the functioning knowledge which allows students to integrate their academic knowledge base. However, the development of this functioning knowledge requires metacognitive activity because students must reflect upon and improve their strategies in this area in order to bring about the magnitude of change demonstrated by their improved LASSI scores. In that sense metacognitive development is evident in the PBL students, who have clearly improved their ability to process ideas by mentally elaborating on them and organizing them in meaningful ways (INP). The PBL curriculum by its very nature requires a different way of using knowledge to solve problems, and it is this functioning knowledge that involves the metacognitive processes in the way that Hmelo et al. (1997) suggests.

The self-regulation component

This component of the LASSI is made up of four items which demonstrate student perception of their ability to focus their attention, develop schedules to manage their time, review their learning, and develop study aids to assist with problem solving. The PBL group also shows significant improvement overall and in comparison to their non-PBL counterparts in all of these areas. Piaget (1977) himself recognised that an environment rich with challenges appropriate to the stage of a child’s development was more important than trying to force the pace of change in order to help increase the pace of cognitive development. Therefore, it should not be surprising to find that metacognitive development also progresses as a result of challenges from the environment and, if these challenges are the result of finding solutions to a real world problem within a fixed timescale, this will involve the internalization of new self-regulatory practices (Downing 2001) and subsequent increases in metacognitive activity as these are further developed and refined. In order to address the problems faced in adapting to a problem-based learning environment, and achieve significant improvements in self-regulation scores, students will need to bring to bear functioning, declarative, procedural and contextual knowledge so that they can integrate the academic knowledge base, develop the professional skills required to solve the problem and learn to control the context in which they develop appropriate solutions.

Everyday challenges emerging from the new social context associated with problem-based learning provide fertile environments for the development of metacognition. The highest ‘meta-level’ of cognition is usually not implicated when we receive an outside task and when the task solution is known. The meta-level tends to be consulted only when things go wrong or when a new problem is confronted. In other words, the challenging new social and academic context of working in the different culture of a problem-based learning environment increases the use of metacognition because the student cannot call upon routinised or ‘automatic’ cognition. Consequently, there is almost a requirement in these circumstances to have knowledge about and control over thinking processes (Allen and Armour-Thomas 1993).

Conclusion

This paper set out to cast light on two main questions. The first was whether problem-based learning would have a significant impact upon the development of metacognition in first-year undergraduates, and the second was the extent to which the use of the LASSI provided an appropriate measure of this development. In considering the first question, this research provides compelling evidence that the gains made by the problem-based group are very unlikely to be a result of chance factors. According to Driscoll (1994), there are three basic instructional principles on which cognitive theorists generally agree:

  1. 1.

    The learning environment should support the activity of the learner (i.e., an active, discovery-oriented environment).

  2. 2.

    The learner’s interactions with peers are an important source of cognitive development (i.e., peer learning and social negotiation).

  3. 3.

    Instructional strategies that make learners aware of conflicts and inconsistencies in their thinking promote cognitive development (i.e., problem-solving and Socratic dialogue).

Why then should metacognitive strategies such as planning, monitoring and evaluating one’s own learning evolve more effectively when undergraduates are engaged in problem-based learning? Vygotsky’s (1986) view was that in order to subject a function to intellectual and voluntary control, we must first possess that function. In other words, metacognition and self-reflection will develop first as a skill before it can be used as a series of consciously controlled strategies and this is precisely what a well-defined and carefully planned problem-based curriculum does. It forces the student into partially unfamiliar territory creating the context for skill development provided appropriate scaffolding and support is available.

In the past research into the problem-based learning has tended to use academic assessment performance as a measure of success (Webster and Riggs 2006; Albanese and Mitchell 1993) and of course this is a crucial factor for all undergraduates and their teachers. However, academic assessment is subject to all sorts of uncontrolled variability and subjectivity and the assessment tasks used to gauge success might vary considerably from one course and one semester to another. In contrast, the LASSI is a standardised and repeated measure which avoids many of these pitfalls and has the advantage of establishing a baseline from which individual and group metacognitive development can be measured. The use of a matched pairs design over a 15 month period also ensures that any practice effects are not only limited but carefully controlled. Therefore, it seems that the LASSI has merit as a measure of the success of problem-based learning environments.

Whilst this research has demonstrated the potential value of both problem-based learning and the LASSI as a measure of metacognition, it is recognised that the nature of the design of the problem-based learning environment is critical to success. If insufficient attention is paid to providing the appropriate levels of scaffolding and support to students then they are unlikely to show the significant metacognitive development illustrated by this study. This is a theme confirmed by Downing (2001) who points out that the extent of the success of any learning process is likely to be due to the same factors that have always been central to the provision of a quality learning experience. These factors include the energy, commitment and imagination of those responsible for providing the learning environment. Unfortunately, assessing these attributes is a far more difficult task.