Introduction

The effectiveness of problem-based learning has been the subject of robust debate for 15 years or more. The debate has been sharpened by the scale of take-up of PBL approaches in many areas of professional education (not just the health sciences), by the associated costs, and by doubts about the nature of the evidence base for this distinctive set of educational practices (Norman and Schmidt 1992; Albanese and Mitchell 1993; Vernon and Blake 1993; Colliver 2000, 2002; Norman and Schmidt 2000; Norman 2003; Dochy et al. 2003; Gijbels et al. 2005; Rikers and Bruin 2006; Colliver and Markwell 2007). One of the subthemes in this debate concerns the ability of educational and psychological theory to guide and underpin PBL practices or to explain observed variations in PBL outcomes (Colliver 2000; Norman and Schmidt 2000; Norman 2003).

The study reported in this paper is part of a larger program of research that gives a central place to the ways in which university students interpret the educational tasks they encounter. Students should not and cannot be treated as unwitting participants in blind trials: their creative and constructive interpretation of task requirements is essential to higher education and cannot be wished away for the purposes of simplistic comparison (Goodyear 2006; c.f. Mol 2002). There has been remarkably little research on students’ conceptions of PBL, or on the interpretive work that they do in making sense of the PBL approach. A notable exception is the recent work reported in this journal by Loyens et al. (2006), which presents persuasive evidence that students’ conceptions of what it means to engage in constructivist learning activities need to be taken into account in understanding the processes and outcomes of PBL (see also, De Grave et al. 2002; Gijbels et al. 2006; Loyens et al. 2007). Our paper can be read as an extension of this line of analysis: but with a sharper focus on some specific learning activities, and the tools on which they draw.

Teachers seeking to prepare graduates for a profession adopt a range of approaches, including the setting of tasks that involve students in research or inquiry activities. Through these activities, students can develop relevant research or inquiry skills as well as a richer conceptual understanding. When coupled with opportunities to master professional problem-solving skills, such as diagnostic skills, this combination of learning opportunities has the capacity to develop robust, applicable professional knowledge: the ‘working knowledge’ of professional practice (Yinger and Hendricks-Lee 1993; Sternberg and Horvath 1999; Goodyear 2006). As we know, teaching approaches are not universally successful and careful research is required to identify the causes of success and failure.

Theoretical background and prior research

There are a number of ways that students’ experiences of learning can be conceptualized. Our chosen theoretical perspective is phenomenography: one of the dominant approaches in research on learning and teaching in higher education over the last 30 years (Marton and Säljö 1976a, b; Entwistle and Ramsden 1983; Marton and Booth 1997; Prosser and Trigwell 1999; Ramsden 2002; Laurillard 2002). This phenomenographic perspective causes us to attend to two aspects of learning: what is learned and how learning takes place (see Fig. 1).

Fig. 1
figure 1

The experience of learning

Full size image

Phenomenography offers researchers a recursive structural/referential model of learning (Marton and Booth 1997, p. 84). It suggests that any phenomenon can be divided into structural aspects (the parts that make it up) and referential aspects (aspects that give it meaning). Over three decades of research into student learning in higher education has revealed the fundamental importance of how students approach their learning and what they think they are learning. Each of these aspects can be recursively re-expressed into structural and referential parts. For approaches, these can be identified as the strategies that students adopt in their learning (structure) and the intention underpinning their strategies (reference). For conceptions, what students learn can be divided into its parts (structure) and its meaning (reference).

Research into student conceptions of learning in higher education has focused on a range of learning situations including writing (Hounsell 1984, 1997), discussions (Ellis et al. 2006), and mathematics (Crawford et al. 1994, 1998). We follow Prosser and Trigwell (1999) and Ramsden (2002) in recognizing the importance of two kinds of conceptions: cohesive and fragmented. In the current context, cohesive conceptions of PBL are those that are closely associated with understanding the principles underpinning the pharmacy problems set, and fragmented conceptions are those in which students do not make a clear connection between the learning activity in which they are engaged and the goal of coming to understand fundamental principles. The division between these two categories is a qualitative one: in which understanding underpins the higher level categories and reproductive aspects of learning underpin the lower categories (Prosser and Trigwell 1999; Ramsden 2002).

Research into student approaches to learning in higher education has occurred in a wide range of disciplines and study situations (Entwistle and Ramsden 1983; Biggs 1987; Prosser and Millar 1989; Ellis 2004). Three types of approaches have been identified in this research: deep, surface and achieving (or strategic). Deep approaches to learning have an orientation towards engaging with the subject matter in ways that promote understanding. Deep approaches are often associated with higher order learning outcomes such as synthesis, integration, critical evaluation and reflection. Surface approaches to learning have an orientation towards reproduction. They often only focus on part of the whole phenomenon being studied. Achieving approaches can exhibit similar strategies to deep approaches. However, since their focus is on short-term performance, an intention to understand deeply is usually absent. This may well mean that long-term knowledge retention rates are poor (Biggs 1987) and that there are limited opportunities for the knowledge to become embedded in professional capabilities (Yinger and Hendricks-Lee 1993; Sternberg and Horvath 1999).

Against a background of strong and sustained interest in PBL (see e.g. Barrows and Tamblyn 1980; Norman and Schmidt 1992; Albanese and Mitchell 1993; Vernon and Blake 1993; Savin-Baden 2000; Colliver 2000; Boud and Feletti 2001; Dochy et al. 2003; Gijbels et al., 2005; Rikers and Bruin 2006) there has been remarkably little phenomenographic research into PBL. Dahlgren et al. (1998) report a small scale study of teachers’ conceptions of PBL indicating that variations in teachers’ ideas about their role are influential in shaping the nature of student activities and also the teachers’ feelings about the success or otherwise of the PBL approach. More recently, Dahlgren and Dahlgren (2002) have illustrated variations in PBL students’ conceptions of their own autonomy, their relationships with their fellow students, and their perceptions of the authenticity of the tasks they have been given. All of these have practical pedagogical consequences. Hendry et al. (2006) developed a questionnaire instrument using closed-ended questions to investigate student conceptions of PBL. This was based on some earlier qualitative research into the student experience of PBL. One of the key outcomes from the Hendry et al. study was that there was no relationship between student conceptions of PBL and their rating of the clarity of the goals and standards of the course. The Hendry et al. study calls for more research into how students approach their learning in PBL courses.

Research aims

In this study, phenomenography is used to investigate variations in, and relationships between, what pharmacy students on a PBL program think they are learning (their conceptions), how they approach their learning (their intentions and strategies), and their academic performance. Our study foregrounds learning tasks that combine face-to-face and online activities. The point here is to look at continuities and discontinuities in students’ conceptions and approaches when they are having to integrate face-to-face and online work. Some of our earlier studies have suggested that integrating old and new kinds of tasks can be a significant source of disruption to students’ working practices and of dissonance in their views and practices (e.g. Ellis and Calvo 2006).

Within this context, we hypothesise that there is a logical connection between what students say they think about learning through problem solving (their conceptions), how they approach learning in situations that combine face-to-face and on-line activity, and their levels of academic performance. The research questions guiding this study can be summarized as follows:

How do pharmacy students approach solving authentic problems when they are expected to integrate a wide variety of on-line information sources with their face-to-face learning activities?

How are the approaches adopted by pharmacy students to solving authentic problems related to their conceptions of problem-based learning?

How are students’ approaches to solving problems in face-to-face and on-line contexts, their conceptions about problem-based learning and their achievements interrelated? What do these relationships suggest about the effectiveness, design and management of PBL for teachers using this approach?

Method

Learning and teaching context for the study

The fourth year undergraduate pharmacists involved in this study are learning the professional decision-making skills associated with medication reviews. Medication reviews involve diagnostic-reasoning processes which allow pharmacists to identify, prevent and resolve potential drug-related problems, with the aim of optimising the use of medicines. Pharmacists can draw on an increasingly complex array of professional, statutory, medical and organizational sources of knowledge. In the university course we investigated, students use problem-based learning (PBL) methods in order to learn how these multiple sources of knowledge should be drawn upon and used in the process of a medication review. PBL was chosen by the teaching team, in part, because of its capacity to help students adopt orderly patterns of reasoning and problem-solving common in authentic professional problem-solving situations.

With this educational outcome in mind, this study investigates how the students approached resolving the problems in class, and how they approached the use of a significant range of professional pharmaceutical resources on-line. (‘Approach’ is used here in the phenomenographic sense of strategy and intention.) In their course, the students had access to a sophisticated range of printed and online information resources for Pharmacy practitioners. These included sources of drug and disease information, such as the Australian Medicines Handbook, Therapeutic Guidelines, Micromedex, eMIMS, and the MERC manual; websites of medical organisations such as The National Heart Foundation Australia; as well as online bibliographic databases such as MEDLINE, EMBASE and International Pharmaceutical Abstracts. These resources offer information about disease states, evidence-based and/or consensus-based treatment guidelines, drug dosage, drug interactions and adverse effects as well as access to the latest research. The online versions of these resources are frequently up-dated and elaborated and some of them include calculation and interrogation tools and other dynamic aids for the professional user. They are regarded as essential sources of information for professional practitioners. Teachers do not want students merely to amass information from these sources. Such activity can produce an unstructured collection of ideas: inert and fragmentary knowledge (Renkl et al. 1996). Instead, they want the students to make judicious choices about which information to use. Judicious choices about the use of information sources need to be integrated with diagnostic reasoning processes as a foundation for making appropriately informed and professional judgments in medication reviews.

The students involved in this research were taking a course that involved 2 hours of lectures, 4 h of tutorials and 5 h of clinical placements (community or hospital pharmacy) per week. This was not their first experience of problem-based learning. They worked on problem scenarios in groups of 8–9 students. Each problem scenario extended over a period of three tutorial sessions, during which students were guided by tutors. In the first session, a new scenario was given to each group. The scenario was designed to prompt students to hypothesise about, and research, the presenting signs, symptoms or product request of a patient and to determine what further information they would require to identify any underlying issues or problems. In the second session, students were given more precise information upon request, such as the patient’s medication history. Based on this information, students conducted further research which allowed them to determine what further information they required or to identify specific problems and possible management options. In the final session, students presented their medication management plan and discussed this with another group of students.

Students were expected to make substantial use of the professional databases and texts available to practicing pharmacists both during and between tutorial sessions. Table 1 outlines some of these key sources of pharmaceutical knowledge.

Table 1 Sources of pharmaceutical knowledge
Full size table

Students were assessed on their demonstrated understanding of the intricacies of putting together appropriate medication management plans in various ways, including their participation in and contribution during tutorials (20%), written medication reviews using cases encountered during their clinical placements (10%) and a final exam based on a medication review (40%). The remaining 30% of assessment was based on reflective diaries, oral presentations, and preceptor assessments related to the students’ clinical placements.

Administration and sample

Ethics approval was sought and received from the appropriate university committee to invite 216 students enrolled in the pharmacy course to engage in the study. Data was gathered in the last 2 weeks of the students’ 14 week semester. The purpose of the investigation was explained to students in their tutorials. Student volunteers for the study were sought and 166 students agreed to be involved: 49 male and 117 female. Draft questions were trialed with five students. After minor adjustments to the questions, 15 students were interviewed and 166 students completed open-ended questionnaires (77% of the 216 students enrolled in the course). The questions used in the questionnaire were the same as in the interviews. The open-ended questionnaire was regarded as an extensive part of the study, since a greater number of students could be surveyed than interviewed. The interviews were regarded as an intensive aspect of the study, allowing the researchers to follow up on student responses, clarifying points made and associated ideas. Interviews lasted around 30 min each. Students interviewed were asked to complete questionnaires as well, in order to triangulate their responses.

Interviews and questionnaires

The main questions used in the interviews and open-ended questionnaires were as follows.

  1. 1.

    What is problem-based learning? (conceptions)

  2. 2.

    How do you approach solving the problem-based scenarios in the class sessions? What do you do and why do you do those things?

  3. 3.

    How do you approach solving the problem-based scenarios using the on-line resources after each tutorial and before the medication review session? What do you do and why do you do those things?

  4. 4.

    When you solve a problem scenario each week, what is your underlying intention when you hand in the final solution to your tutor?

Question 1 investigates student conceptions of problem-based learning. Analysis of their responses reveals variation in what they think they learn through PBL. Questions 2 and 3 investigate student approaches to PBL. The sub-questions probe ‘what’ they do (the strategies they use in the approach) and ‘why’ they do it (the intentions that underpin their approach in each context). Question 4 investigates the intention underlying the submission of the solution to the problem each week.

During the interview process, student answers were probed to try to elicit the underlying meaning of what the students were saying. Students were asked to re-express ideas and words stressed by them as important, as a way of trying to understand what was meant. This helped unpack the different meanings that different students attached to the same words. The open-ended questionnaire was used to survey a greater sample of students so that the researchers could draw on a wider base of student experiences of problem-based learning, to improve the analyses and the robustness of the categories derived from what the (smaller number of) students said in interviews.

Method of analysis

The responses to these four questions were analysed by three members of the research team following a phenomenographic approach. The process of analysing the student responses is described below, using question 3 as an example.

  • All the student responses to question 3—how they approached using the on-line resources—were read, to get a feeling for the variation in the approaches described.

  • As the researchers read and discussed the responses, key themes began to appear in the data, some suggesting an integrated use of the sources of information, and others suggesting a collecting-additive use of the information.

  • The themes were shared amongst all researchers and were grouped into logically related areas. Some of the themes overlapped and were subsumed under other themes which appeared to be operating at a more inclusive level. These higher-level themes began to form the ‘outcome space’ for on-line approaches to solving the problem-based scenarios (Prosser and Trigwell 1999).

  • All the students’ responses were then re-read to see if they fell within the draft outcome space being identified. This represents the initial classification of the student responses.

  • The initial category descriptions required further development as the researchers discussed the nature of the variation. To help the researchers decide on the final structure of the categories, 25 questionnaires were chosen at random and were categorised by the researchers independently using the draft outcome space. Using the categorization of one researcher as a baseline, the percentage agreement of the two other researchers classifying the student responses is shown in Table 2. The table shows there was between 90% and 100% agreement for the classifications after consultation.

  • In addition, to improve the clarity of the analysis process, illuminative extracts from the student responses were chosen. These are presented in Tables 3–5 below.

  • The redrafted categories became the final version shown in Tables 3–5. The draft categories and the illuminative quotations draw on the SOLO taxonomy for their underlying structure and hierarchy (Biggs 1999). The SOLO taxonomy can be used to describe a set of hierarchical and logical relationships amongst student learning outcomes. Examples of this are shown in Tables 3–5.

Table 2 shows the percentage agreements amongst the three researchers involved in the categorization of the questionnaires.

Table 2 Inter-researcher agreement over categorizations before and after consultation
Full size table

Results

We present a description of the main categories of conception and approach emerging from our qualitative analysis of the students’ interview transcripts and questionnaires (Tables 3–5). Each table shows a hierarchy of categories and gives illustrative quotations to help convey the nature of the categories. After that, we present the outcomes of a quantitative analysis of the distribution of students across categories, and of associations between conceptions, approaches and course marks (Tables 6–9).

Table 3 Conceptions of PBL in pharmacy
Full size table

Table 3 shows categories of conceptions of problem-based learning in Pharmacy; that is, the different ways students’ reported conceiving of PBL. Table 3 identifies six categories of conceptions. The categories are hierarchical and inclusive. The first two categories (A and B) emphasise the understanding that is required to resolve problem-based scenarios. In this sense they are cohesive categories (Prosser and Trigwell 1999). Category A is more holistic or encompassing than Category B. It acknowledges that the professional context for resolving Pharmacy problems provided by PBL requires independent reasoning and problem solving. Category B shows an awareness of the complexity of understanding required to resolve Pharmacy cases, but without foregrounding the professional context from which they come. In contrast to these two categories, the remaining categories do not place any emphasis on understanding. In this sense, they are fragmented categories (Prosser and Trigwell 1999; Ellis 2004). Category C shows an awareness of a process of learning, but it is more like a rehearsal than learning that endures. Category D suggests that PBL is only about finding answers. Category E suggests that PBL is about following a formula, and Category F suggests that PBL is mainly about developing generic transferable skills, such as teamworking skills.

Table 4 presents categories of approaches to solving problem-based scenarios in face-to-face contexts.

Table 4 Approaches to problem-based scenarios face-to-face
Full size table

Table 4 identifies five categories. The categories are hierarchical but not inclusive, as the following descriptions will show. Category A is a deep approach. It emphasises a need to use professional methodologies and judgment in order to fully understand the problem scenarios. Category B is an achieving approach. It emphasises a deep strategy to understand the context of a patient’s situation with a main intention of performing well in the assessment of the case. Categories C–E are surface approaches to solving problem scenarios face-to-face. They are not underpinned by an intention to fully understand the problems. Category C emphasises gathering information, category D emphasises routine work and category E emphasises a main purpose of developing generic skills without being aware of their particular relevance to Pharmacy contexts.

Table 5 presents the categories of approaches to researching problem-based scenarios on-line.

Table 5 Approaches to researching problem-based scenarios on-line
Full size table

Table 5 presents five categories. Like the approaches in Table 4, they are hierarchical but not inclusive. Category A is a deep approach to researching on-line. It emphasises a use of on-line pharmaceutical databases in ways that promote diagnostic reasoning. Category B is an achieving approach. It emphasises a use of on-line pharmaceutical databases in ways that promote understanding with a main intention of performing well in assessment. Categories C–E are surface approaches and are not underpinned by an intention to fully understand the problem scenarios. Category C emphasises finding information in databases, category D emphasises finding answers and category E emphasises using databases only if they are easy to use.

We now turn to a quantitative analysis of the distribution of students across categories and of associations between conceptions, approaches and course outcomes. The overall pattern of responses is shown in Table 6.

Table 6 Structure and distribution of conceptions and approaches
Full size table

Table 6 has four columns. Column 1 identifies the categories of conceptions and approaches to problem-based learning. Column 2 shows the distribution of the 166 sets of responses across all categories and columns 3 and 4 show the percentage breakdowns.

When reading the categories of conceptions and approaches, it should not be assumed that there are direct correlations between all categories labeled A or all categories labeled B for example. Rather, the relationships amongst the categories are more subtle and complex, some statistically significant, and others not, as Tables 7–9 show. Table 7 identifies associations amongst conceptions and approaches to PBL in this research context.

Table 7 Relationships amongst conceptions and approaches
Full size table

Table 7 combines two 2 × 3 contingency tables. The upper 2 × 3 table shows conceptions of problem-based learning (cohesive, fragmented) in rows, and face-to-face approaches to problem-based learning (deep, achieving, surface) in columns. The lower 2 × 3 table shows conceptions of problem-based learning (cohesive, fragmented) in rows and on-line approaches (deep, achieving and surface) in columns. Overall, the associations amongst face-to-face approaches and conceptions are significant, while the associations amongst on-line approaches and conceptions are not.

To understand which of the cells in the first 2 × 3 contingency table are responsible for the significance of the results, a series of 2 × 2 cross tabulations were conducted, sequentially excluding from the analysis deep, achieving and surface categories. The results reveal that a deep approach to PBL tends to be associated with a cohesive conception while a surface approach tends to be associated with a fragmented conception of PBL (for 2 × 2 table, χ 2 = 5.139, phi = .195, p < .05).

Table 8 shows associations amongst face-to-face and on-line approaches to problem-based learning. Table 8 is a 3 × 3 contingency table showing face-to-face approaches to problem-based learning (deep, achieving, surface) in columns and on-line approaches (deep, achieving and surface) in rows. Overall, the associations amongst face-to-face and on-line are significant.

Table 8 Relationships amongst approaches to PBL in pharmacy
Full size table

To understand which of the cells in the 3 × 3 contingency table are responsible for the significance of the results, a series of 2 × 2 cross tabulations were conducted, sequentially excluding from the analysis deep, achieving and surface categories. The results reveal that a deep face-to-face approach to PBL tends to be associated with a deep on-line approach while a surface face-to-face approach tends to be associated with a surface on-line approach to PBL (for 2 × 2 table, χ 2 = 47.977, phi = .605, p < .001).

Table 9 shows relationships between conceptions and approaches and student performance as measured by final course mark. Only those variables showing a significant association with final mark are included in the table.

Table 9 Relationships between conceptions, approaches and performance
Full size table

Table 9 shows that students who held a cohesive conception of PBL tended to perform at a higher level than students holding a fragmented conception (t = 4.6, p < .001, es = .35). It shows that students who described taking a deep face-to-face approach to PBL tended to perform at a higher level than students adopting an achieving or surface approach (t = 2.2, p < .05, es = .28; t = 3.5 p < .05, es = .35). It also shows that students who described taking a deep on-line approach to PBL tended to perform at a higher level than students adopting a surface on-line approach (t = 2.5, p < .05, es = .28). These are medium effect sizes. There was not a significant difference in final mark between students taking deep and achieving approaches in their use of online tools and resources.

Discussion

Limitations of the study

Before discussing the outcomes of this study, we should note some of its limitations. The results are based on the student experience of one undergraduate fourth year pharmacy class engaged in PBL, using interviews and open-ended questionnaires with a population of 166 students. The phenomenographical investigation of the PBL pharmacy experience is an evaluation from a student perspective, rather than a tutor or group perspective. In other words, the study considered what the students think they learn through PBL (their conceptions, Table 3), how they approach their learning in class (face-face approaches, Table 4,) and using the internet (on-line approaches, Table 5), and how these are related to the marks they received (their academic performance). It is not a study of the effectiveness of the PBL experience from the perspective of a tutor (c.f. Dahlgren et al. 1998). Future studies may involve links between student and tutor experiences of problem-based learning. Nor does the study address longer-term issues of professional competence or the appropriateness of the forms of PBL used on this course as a preparation for professional practice (c.f. Dahlgren et al. 2006). Having acknowledged this, the authors believe the student perspective is sufficiently important to warrant serious attention in its own right.

Summary of results

Part of the point of this study has been to show that students’ experiences and intentions are critical variables in the complex web of influences upon the success or failure of an educational approach like PBL. Research or evaluation strategies that try to ‘blind out’ these key participants in the educational process are themselves heading up a blind alley.

More specifically the study explored the conjecture that the way students approached solving problems face-to-face and on-line, the way they conceived of what they were learning through PBL and their levels of achievement were related.

The results showed that deep approaches face-to-face were positively related to deep approaches on-line (Table 8). In addition, deep approaches face-to-face were positively related to student conceptions of learning that showed an awareness of the importance of resolving problems through clinical reasoning (Table 7). Positive associations were also found between conceptions, face-to-face approaches and on-line approaches and performance (Table 9).

No associations were found in this study between cohesive conceptions and on-line approaches. This may be because of the newness of the medium or the complexity of the relationships between beliefs about learning, study strategies and the use of learning resources (c.f. Te Winkel et al. 2006). One of the purposes of the study was to identify whether and how student approaches to learning on-line complemented their approaches to learning in class. If we do not understand how their on-line experiences relate to the learning activities in class, then we will not be in a position to make the most effective use of the on-line part of their experiences as a way of extending and elaborating how they learn face-to-face. Student approaches to solving the problems on-line varied qualitatively (Table 5) and in frequency (Table 6). Deep approaches on-line (categories A and B) involved a comparison and synthesis of the pharmaceutical knowledge in the databases to arrive at a more considered answer to resolve the problems. Surface approaches on-line (C, D and E) were comparatively more narrow in their approach, focusing on reproductive, and formulaic strategies such as finding information rather than engaging more critically with the ideas they were discovering.

In summary, students who reported experiencing PBL as a way of developing independent clinical reasoning and problem solving, who approached their studies in class as a chance to use professional methodologies and judgment in resolving the cases and who used the on-line resources as a way of deepening their understanding of the specifics of cases, tended to perform at higher levels. An equally important result was that students who reported experiencing PBL as a way of rehearsing being a pharmacist, by gathering information from others and using databases to find answers, tended to perform at relatively lower levels. These results have significant implications for teachers.

Implications for teaching and further research

For teachers, the results suggest that students tend to perform at relatively higher levels in PBL situations when they approach problems to narrow possibilities in relation to context, when they avoid surface strategies such as jumping to conclusions too quickly, when they approach the use of professional pharmaceutical databases to cross-reference their ideas, to delve more deeply into the research-base of advice and to understand that different databases offer different types of support. Such approaches were found to be related to a way of thinking about PBL that emphasised joining up key parts of a case and its elements, and that connected with the development of a professional decision-making process that closely resembled clinic-based reasoning. It is not just the identification of these key aspects of PBL in pharmacy that is important for teaching. Rather it is the links between them that may help teachers to a better understanding of what is happening and how to guide student activity. On the one hand, teachers could integrate activities that encourage debate amongst students about how a problem-solving experience can help their professional development. If teachers are successful in helping students to adopt more cohesive conceptions of PBL, then the links with approaches suggest that students are more likely to approach PBL in helpful ways. On the other hand, teachers could begin with activities that scaffold how students approach PBL in class. The pairwise associations amongst the categories in Table 7 would suggest that the development of a sound approach may also help students to develop more useful conceptions of PBL, propelled by the powerful forces of dissonance reduction. In either case, an improvement in conceptions of PBL or approaches to PBL would suggest higher levels of performance, judging by the results in Table 9.

For researchers, the complementary roles of face-to-face and on-line experiences in the research design has interesting implications. The dominant tradition in evaluations of online learning has been one of comparing alternatives, despite a long history of studies in educational technology reporting no significant differences (Clark 1994). Finding ways to consider their combined contributions is a challenge. Clearly, the student approaches to learning in face-to-face situations in class and using on-line resources are different in this study. In class, approaches to solving problems require the application of professional-like judgment to understanding the case and its unique issues. On-line, it requires a judicious use of an array of professional tools such as databases—in turn depending upon understanding what light each resource can shed on the medication review being studied. The classification of student approaches to researching problem scenarios on-line is the first analysis of its kind in pharmacy of which we are aware. The quantitative analyses of the qualitative categories show that deep approaches on-line were related to deep approaches in face-to-face contexts. On-line approaches are different in structure to face-to-face approaches, as Tables 4 and 5 show, but the underlying qualitatively different referential aspects identified in the shifts between categories A/B and categories C/D/E are similar. In this study, deep approaches on-line were not significantly related to cohesive conceptions of problem-based learning, probably because they are a relatively new part of the learning experience and students are yet to fully understand how to integrate them into their experiences of learning. With the increase in the use of on-line sources of knowledge for professional work, this is an area that requires urgent research.

For both teachers and researchers, an interesting category of approaches—achieving approaches—was identified in this study. Despite being motivated by the desire to achieve higher marks, these approaches were not significantly related to higher performance. This provides valuable evidence to teachers who want to convince students that intentions to understand the subject matter will result in higher levels of performance than intentions primarily driven by marks. When Biggs (1987) and colleagues (Biggs et al. 2001) discussed deep, achieving and surface approaches to learning, they theorised about their differences. Surface approaches are those adopted by students who want to ‘just get by’. Achieving approaches are adopted by students who put achieving high marks above everything else. Deep approaches are adopted by students who want a fundamental understanding of the concepts first and foremost. In university contexts where competition is high amongst high-achieving students (such as those in Pharmacy), where education is expensive and the stakes based on success in one’s studies are great, it is often a real challenge to help some students see beyond the immediate satisfaction of a good mark to the deeper satisfaction of real understanding and the longer-range goal of professional competence. This appears to be a challenge for students adopting an achieving approach in this study. There is also the challenge of motivating those students who want to ‘just get by’ to strive for higher levels of academic performance underpinned by understanding. The results of this study are encouraging for teachers and researchers hoping to help students approach their learning in familiar and new contexts of learning in ways that really go to the heart of the matter and improve understanding.