Introduction

Problem based learning (PBL) is an educational strategy in which students are encouraged to take responsibility for their own learning. In the medical profession, the ability to direct and regulate one’s own learning experience is crucial to success (Mast and Davis 1994). Self-directed learning (SDL) skills, which are associated with lifelong learning (Candy 1991), present an attractive solution to the ceaseless growth of biomedical knowledge for developing graduates who can sustain the educational demands of continuing professional development. According to Knowles (1975), “self-directed learning describes a process in which individuals take the initiative, with or without the help of others, in diagnosing their learning needs, formulating learning goals, identifying human and material resources for learning, choosing and implementing appropriate learning strategies, and evaluating learning outcomes.” Norman and Schmidt (1992) argued that PBL appears to have a large and potentially long-lasting impact on self-directed learning skills.

Although the available evidence is not conclusive, there is some evidence to support the assumption that PBL encourages SDL (Blumberg and Michael 1992; Ryan 1993; Dolmans and Schmidt 1994). These studies concluded that when students are responsible for their own learning, they acquire autonomous learning skills, and learn to be better self-directed learners as they progress through their studies. Blumberg (2000) reviewed the research evidence and concluded that PBL students are active library users, they employ deep-level study strategies and they perceive themselves as continuing to improve their own SDL abilities. A recent study comparing PBL and traditional curricula reported that PBL students showed significantly more self-regulated learning and they perceived themselves as more active contributors to group learning process and used a broader range of resources than students in the traditional programme (Lycke et al. 2006). In a study of graduates since 1980 of a PBL and a conventional medical school, PBL graduates rated themselves more highly on 14 of 18 criteria, including self-directed learning (Schmidt et al. 2006). A systematic review of the effects of PBL during medical school on physician competency reported a moderate level of evidence in support of PBL for self-directed continuing learning, when compared with traditional curricula (Koh et al. 2008).

In contrast to the above optimistic view of the effect of PBL on SDL, there have been some concerns in relation to the notion of self-directed learning. Miflin et al. (1999) found that instead of developing self-direction, students became overly dependent on faculty direction in the first year of their new PBL curriculum. A second study, done in an integrated PBL curriculum in United Kingdom, reported that student learning was not self- directed; rather it was socially agreed amongst the peer group and directed by resources provided by faculty (Lloyd-Jones and Hak 2004). Harvey et al. (2003) reported that they found no evidence that students’ self-reported SDL is positively influenced by the hybrid curriculum at the Faculty of Medicine, University of Toronto.

The inconsistent findings about PBL and its effect on SDL may be explained by several factors including the curriculum itself, whether described as ‘pure’ PBL or a more hybrid curriculum, and the level of shared understanding of what SDL means between stakeholders, especially students and faculty, etc. Most of the evidence supporting PBL in fostering SDL has been reported by schools with ‘pure’ PBL curricula. Evidence from integrated or hybrid PBL schools are rather inconsistent. Therefore, more evidence is required to understand whether PBL actually leads to students becoming better self-directed learners, especially in a hybrid PBL curriculum, which has become quite common, incorporating relevant contextual features of both PBL and traditional curriculum.

Dalhousie University Faculty of Medicine in Canada introduced a new PBL curriculum known as the case-oriented problem-stimulated (COPS) curriculum in 1992 (Mann and Kaufman 1995). The framework of the COPS curriculum combines support for learning and encouragement of student responsibility based on the evolving needs of students. In contrast to an entirely problem-based curriculum, the curriculum includes tutorials, along with resource sessions such as lectures, labs, and a clinical skills learning program, all of which are intended to differently and cooperatively affect students’ learning. Former studies of the preclinical PBL curriculum in the COPS program have supported that students’ perception of the curriculum as positive, relevant and challenging, and requiring active student involvement in learning (Kaufman and Mann 1996; Mann and Kaufman 1999). However, comparisons of self-reported, self-directed learning have not revealed significant differences between students in the PBL and conventional curricula (Mann and Kaufman 1994). In addition, research exploring how the constituent components of COPS program affect the student’s self-directed learning and the relationship between the various components has not yet been conducted.

The goal of this study was to investigate the influence of the COPS curriculum on students’ self directed learning. The specific research questions aimed to determine: (1) how students assess their own SDL ability and whether this differs between year 1 and year 2 students, (2) student perceptions of the influence of curricular components on SDL and whether this differs between year 1 and year 2 students, and (3) the nature of the relationships that exist between the curricular components and SDL.

Methods

Context and participants

The study was undertaken at Dalhousie University Faculty of Medicine in Canada. The curriculum known as the COPS (case-oriented problem-stimulated) is a hybrid PBL curriculum. The two preclinical years are based mostly on real (or modified) paper patient cases, prepared collaboratively both basic science and clinical faculty. The cases are developed to reflect the core objectives of the preclinical years. Cases are primarily intended to serve as a stimulus to learn the required basic, clinical and behavioral science content. Course units follow each other serially; each including tutorials, lectures and laboratories, along with patient–doctor unit and population health units, and an elective experience. Student achievement was assessed via pass-or-fail examination.

Following ethical approval, all year 1 (n = 93) and year 2 (n = 93) medical students were invited to participate in the study. A questionnaire was administered to students at the end of Unit 2 of each of year 1 and year 2. In February and March of 2004, an electronic questionnaire survey was conducted with eligible participants. For the two focus group interviews, 10 medical students from each of year 1 (n = 5) and year 2 (n = 5) participated. In addition, eight experienced PBL tutors participated in the faculty focus group interview. All participation in surveys and interviews was voluntary and based on informed consent.

Instruments

The research questions were explored both quantitatively and qualitatively, using a mixed methods approach (Creswell 2003).

Questionnaire development

The survey instrument was developed based on a literature review, the input of PBL experts and the results of the pilot test. The three—part questionnaire was constructed as follows; (1) Part I: demographic data, (2) Part II: students’ assessment of their self-directed learning ability, and (3) Part III: perceived influence of various curriculum elements on individual study. Part I consisted of 4 items including gender, age, the final degree before entering medical school and small group learning experience prior to medical school. For Part II and Part III items, students were asked to indicate on a 5-point scale whether they (1) totally disagree, (2) disagree, (3) are neutral, (4) agree, or (5) totally disagree with each statement.

To develop the items exploring students’ self-assessment of their SDL (Part II), the authors conducted a literature review (Knowles 1975; Candy 1991) and defined SDL as follows: students’ ability to (1) self-assess and identify their specific learning needs; (2) plan and operationalize learning; (3) identify, use, and access a range of relevant resources to address their identified learning needs; (4) evaluate the scope and accuracy of the information, and (5) evaluate the effectiveness of the learning process. Based on this definition, we developed 14 items (each on a 5—point Likert scale) to explore students’ perceptions of their own SDL ability.

The questions identifying the influence of curricular elements on students’ SDL(Part III) were adapted with permission from a questionnaire developed by Dolmans and Schmidt (1994), which consisted of 20 items assessing the adequacy of the six elements influencing students’ decisions on what to study: discussion (4 items), content tested (10 items), course objectives (3 items), lectures (2 items), tutors (3 items), and references (2 items). Dolmans and Schmidt conducted a confirmatory factor analysis and showed the six-factor model was a reasonable fit to the data; the coefficient alpha for each factor varied between .51 and .82. Building on Dolmans & Schmidt’s questionnaire, the authors of the present study added 14 additional items . A 34 item questionnaire resulted, which was divided into 6 themes: influence of the tutorial discussion (7 items), the influence of content tested (7 items), the influence of unit/case objectives (7 items), the influence of lectures (3 items), the influence of tutors (3 items), and the selection of resources (7 items). Two experts in medical education (co-authors of this study) and three experienced PBL tutors at Dalhousie Medical School reviewed the questionnaire to evaluate content validity. They provided opinions and comments to ensure that items accurately reflected the COPS curriculum and that all domains of activity were represented.

A pilot test using an electronic questionnaire survey was completed by six volunteer year 3 medical students. Revisions were made, based on their feedback.

Focus group interviews

Two student and one faculty focus group interviews were conducted after the survey. We conducted in-depth, semi-structured group interviews. The interview protocol was developed based upon the information gained through the surveys; however, in general, the interview was focused on the influence of the components of COPS program on participants’ self-directed learning. The interview questions addressed the following areas: (a) the most important influence on individual study in the COPS curriculum; (b) how the participants defined/described SDL; (c) how participating in the COPS curriculum had affected students’ SDL ability; (d) the influence of the end-of-unit examination on individual study and SDL; (e) how to relate each element of the COPS curriculum to guide learning, and (f) how the tutorial process, including the group, cases, tutors etc., influenced individual study and self-directed learning. The same interview questions were used for both student and faculty groups.

All data from the interviews protected the identity of the participants via pseudonyms. The interviews were audio-tape recorded and transcribed.

Analysis

Quantitative data were analyzed using SPSS 12.0 K for Windows. Exploratory factor analysis using principal axis factoring and Varimax rotation were conducted for the items in Part II and III of the questionnaire. The internal consistency of the scales was calculated using the Cronbach alpha statistic. Eight out of 14 items of Part II (students’ self-assessed SDL ability) converged to a single factor; the reliability coefficient was .82. The items of Part III were extracted into six factors. The six factors and the reliability coefficient of each factor are shown in Appendix 1. The Mann–Whitney U-test was performed to compare the differences in scores across the year of study. The relationship between the curricular components and SDL was evaluated by Spearman’s rank correlation analysis.

The data from the transcribed interview texts were sorted initially and open coded and primary categories of information were formed by segmenting information. The team engaged in axial coding of the information, re-assembling the data to identify central phenomena, causal conditions, specific strategies, and context. Following axial coding, the team selectively coded the data to integrate the categories from the axial coding model (Creswell 1998).

This study was conducted in accordance with the Canadian Tri-Council Policy Statement on Research Ethics and was reviewed by the Dalhousie University Health Sciences Ethics Review Board.

Results

The survey results

The overall response rate was 64% (119/186):72% of year 1 and 56% of year 2 students participated in the survey. The demographic information of respondents is shown in Table 1.

Table 1 Respondent demographics
Full size table

Students’ self-assessed SDL ability

Table 2 shows the overall mean scores of SDL ability assessed by students. Students rated their SDL skills highly, particularly identifying knowledge deficits (X = 4.06/5; SD 0.74), learning skills and strategies (X = 4.03/5; SD 0.82), and managing study time (X = 4.05/; SD 0.79). The mean scores of year 2 in defining learning needs, managing study time, identifying resources, and evaluation of resources, were higher than those of year 1, but the differences were not statistically significant (P = .852).

Table 2 Students’ assessed self-directed learning ability
Full size table

Students’ perception of the influence of each curricular component on their individual learning

Table 3 indicates the students’ perceived influence of each curricular component on their individual learning. Students rated the influence of lectures on individual study the most highly; they thought lectures helped them to select study topics and learn for the tutorial case (see Appendix 1). Other components including tutors, unit/case objectives, tests, and tutorial discussions, were also regarded as factors influencing what to study and learning process. Regarding resource seeking, students seemed not to be influenced by the suggested materials or cases; rather they freely used the information they needed. No significant difference of response was found between years 1 and 2 students.

Table 3 Students’ perception of the influence of the COPS curriculum on their individual learning
Full size table

Relationships between the curricular components and students’ perceptions of SDL ability

Relationships among the six components of the COPS and students’ self-assessed SDL ability, were analyzed. Among the six components, tutorial discussion and unit/case objectives showed a weak correlation with SDL ability (Table 4). In addition, tutorial discussion was moderately correlated with unit/case objectives and weakly correlated with tutors.

Table 4 Correlations among the six components of the COPS and self-directed learning ability (N = 109)
Full size table

Focus group interview results

The overall impact of the COPS curriculum on students’ SDL ability

The students from both years seemed able to define clearly the concept of self-directed learning, which for them, included the ability to (1) identify what to learn and recognize personal knowledge gaps; (2) seek and use appropriate resources to meet the identified learning needs; (3) evaluate the effectiveness of learning process, and (4) take ownership of their learning. Illustrative quotes are found below.

Definition of self-directed learning has certainly changed since getting into a program where it is focused. I think I originally thought it was literally spending a lot of time learning, … But, I think now self-directed learning is really the ability to generate your own questions, whether it is case-based, or lectures, or being in the hospital.

Knowing what your resources are; how to use the resources that you have; finding reliable resources

The overall influence of the COPS curriculum on students’ self-directed learning abilities was positively perceived by both years. However, year 2 students revealed more elaborated thoughts and deeper insights; their SDL has been improved through the fostering of active learning and personal responsibility, critical analysis of effectiveness of information, and increasing sociability and engagement of peers. The descriptions below were expressed by year 2 students

I would say it is safe to say that all our self-directed learning has improved

I have become more critical in my evaluation of resources … So, how it has affected my ability, I would say it has increased my ability to critically evaluate information

The influences in the COPS curriculum on students’ perception of their SDL ability

Expectations by peers and group dynamics were revealed as important factors influencing SDL in both years. Personal intellectual curiosity or desire to have a strong foundation or deepening of knowledge, and professional responsibility reinforced by case, were identified as important influencing factors on SDL.

Assigned a learning issue within the group, being the facilitator or teacher within the group fosters ownership for the content: I find out where the deficiencies are in my knowledge when I have to teach somebody else about that concept and they start asking me questions back, and that is a huge part of the tutorial group.

The end of unit examination was mentioned as an important influence on individual study, although there were some different views between year 1 and year 2. Year 1 students were more unanimously positive about the effect of the examinations, and they appreciated the pass/fail evaluation system in terms of encouraging their SDL as shown in the following quotes.

I think having pass-fail is really important.… I think it is easier to be less focused on the exam and more focused on learning what is clinically important or interesting to you because it is pass-fail

In contrast, year 2 students equally argued the positive and negative influence of the exam on SDL.

The influence of the exam on SDL would be different across the years; if there (were) is no exam, at the beginning of the year 1 would not survive, but now I can survive without exam. And if in second year there weren’t exams, it would be O.K

The threat of the exam of the end of the unit motivates me to study in a way that I don’t find is necessarily constructive. So, for example, memorizing details and numbers, while I then tend to skip over broad concepts …

Relationships among the elements of the COPS curriculum in facilitating students’ SDL

Students mentioned that the COPS curriculum provides a diverse learning environment and their learning can be consolidated in different teaching contexts. They noted, however, that synergism among the curricular elements can be best achieved when the synchronization, sequencing, and content relevancy are aligned in a topic or within a unit.

… not everybody learns the same way, so maybe somebody needs to be able see in lab, see a chest x-ray,…, where somebody else can understand from a textbook description … It (COPS) gives you a lot of different ways of approaching the same topic

I think you can really tell and people appreciate it whenever they are in synchronization…it is really obvious whenever it is planned well and it works out

Students expressed that successful tutorials depend on the joint influences of positive group dynamics, the role of the tutor, the case and the strong student initiative. The tutor was also identified as an important influence on individual study and self-directed learning in the COPS curriculum. There were balanced arguments in support of expert vs. non-expert tutors; students also expressed that tutors must be well trained to be good facilitators and more standardized, detailed evaluations are needed.

The group, the tutors, the case, they really, really, really influence for me how much individual study I need to do, what kind of individual study I need to do and how much self-directed learning is broad or focused.

Faculty focus group interview

Most of the findings from the faculty group interview supported the results of the students’ focus group interview. Faculty valued the COPS curriculum in term of fostering students’ learning and SDL. Faculty argued that synchronization, coordination, and organisation of each element within COPS curriculum were critical in facilitating students’ learning. They regarded collaboration and discussion through the tutorials as a strong influence on individual learning and self-directed learning. In addition, faculty mentioned the importance of the student selection process and the type of students within the COPS program. They considered most students as highly motivated, independent, and competitive even prior to their exposure to the COPS curriculum.

Discussion

Evidence supporting whether a hybrid PBL curriculum actually leads to students becoming better self-directed learners is inconclusive, compared to evidence reported by pure PBL curricular schools. We investigated the influence of the COPS curriculum, a hybrid PBL curriculum, on students’ self-directed learning.

The students rated their SDL ability relatively highly, but there were no significant differences in scores across the years. These survey results were similar to the previous study by Mann and Kaufman (1994), who reported no difference in self-directed learning readiness scores in a group of Dalhousie University medical students between their first and second years in a hybrid PBL curriculum. Those authors explained the findings by suggesting that the curricular structure is similar in both years and students may learn many of the required self-directed learning skills by the end of year 1. Another Canadian study (Harvey et al. 2003) reported that no significant trend in SDL was evident by curriculum year; in fact, there was some evidence of decreasing SDL in senior years. In contrast, Dolmans & Schmidt suggested that students in a problem-based curriculum become more accomplished self-directed learners over the four curriculum years (Dolmans and Schmidt 1994). Although there were no significant inter-year differences in students’ perceptions of their SDL ability in our study, the findings of the focus groups showed that students thought their SDL ability was positively influenced through the COPS curriculum.

In exploring how each curricular component affects students’ SDL, students rated the influence of lectures on their individual studies most highly. But it does not necessarily follow that students are more dependent on lectures than other curriculum components. In focus group interview, students mentioned that the COPS curriculum provides a diverse learning environment and their self-directed learning can be consolidated and reinforced in different teaching contexts such as tutorials, lectures, and clinics. The perceived effects of tests on students’ individual learning, and the dependency on references suggested by the case or unit materials were lower in year 2 students than year 1, although the difference was not significant. Dolmans and Schmidt also reported that first-year students tend to rely more on the literature cited in the references list and content covered in lectures and tests than students in the other three curriculum years and the influence of these elements decreased over the four curriculum years (Dolmans and Schmidt 1994). The fact that our study did not show a difference across study years may be explained in part by the fact that this study was conducted at the end of the first semester, at which point students were already becoming accustomed to the study habits necessary to survive in the COPS curriculum. Across both years, our results support that the several curricular components in a unit cooperatively and positively influenced students’ self-directed learning. This result is in keeping with the goals of a hybrid curriculum, and may reflect that there is some coherence to the students’ experience of the curriculum.

In our study, tutorial discussion and unit/case objectives showed a weak correlation with SDL ability, and tutorial discussion was moderately correlated with unit/case objectives and weakly correlated with tutors. In contrast, while the lecture was viewed as directing students as to what to study, it was not correlated with perceived SDL ability. This suggests that the elements of the curriculum influence learners’ differentially. The lecture is actually intended to frame the students, learning, while tutorial discussions and tutorial group work influence what students identify as their SDL activities. The positive influence of the tutorial discussion on learning was also apparent in the focus group results. However, we did not examine the causal relationship between curricular components and SDL ability, and therefore could not conclude which specific component of the COPS curriculum was most influential on students’ SDL. Some previous studies (Schmidt and Moust 1995; Van Berkel and Dolmans 2006) reported strong evidence of the influence of the tutorial discussion on SDL using a causal path between the elements in PBL curriculum. Further investigation is needed to better understand the relationship between the curricular components and their effect on self-directed learning.

There are several limitations of this study. The response rate to the questionnaire was 64% of total expecting participants. Therefore, the results of the study may not be representative of all students’ perspectives on the study questions. Because SDL ability was assessed by students themselves, it may not reliably reflect their true SDL capabilities. Although the focus group results showed that the students could clearly define the concept of self-directed learning and SDL ability, it does not guarantee that all respondents assessed their SDL ability in reliable way. We did not find that perceived SDL ability increased across the years. However, our study was a cross-sectional study and we studied only preclinical years, not the entire four years of students. Some other studies have reported a favorable influence of preclinical PBL experience on clinical performance during clerkships (Distlehorst et al. 2005), and furthermore, long-term effects on clinical competencies of physicians (Schmidt et al. 2006; Koh et al. 2008). Clearer understanding of the effect of hybrid PBL curricula on students’ SDL could be achieved by a longitudinal follow-up assessment.

Conclusion

Students in this study evaluated their self-directed learning ability relatively highly. Although we could not find significant improvement in perceived SDL ability between years 1 and 2, several components of the COPS curriculum were perceived as positively influencing students’ SDL. In addition, students were able to describe the influence of these factors on their SDL. This study found that each component of the COPS curriculum, which is a hybrid PBL curriculum, cooperatively and positively affects students’ SDL. Further investigations are needed for a clearer understanding of the specific effects of the hybrid PBL curriculum on students’ SDL.