Introduction

There has been a global move for change in medical education with key documents coming from the Association of American Medical Colleges, the General Medical Council in the U.K. (2009) and from the Irish Medical Council [1–3]. In response our School of Medicine at the National University of Ireland, Galway introduced a new 5-year curriculum which is a student-centered, systems-based curriculum with emphasis on integration, interactive learning and professionalism. The new curriculum consists of a core curriculum and electives and was introduced in September 2006 with the enrolment of students into year 1 of the 5-year program. By contrast, the old curriculum was a 6-year, Flexner-style curriculum, which was divided into pre-clinical and clinical years, where disciplines were separated and much of the teaching was didactic. With the introduction of a new curriculum parallel changes take place in the environment, indeed “the environment derives from, and is a manifestation of the curriculum” [4]. Analysis of the educational environment gives us valuable information about the satisfaction of students with the delivery of the curriculum, the quality of the learning experience and the learning atmosphere. Furthermore students’ perceptions of the climate of the educational environment are related to their achievements and satisfaction with the course [5–7].

The Dundee Ready Education Environment Measure (DREEM) is a 50 item validated test, which has been shown to have reliability across culturally diverse student samples [8]. It can be used to assess both the whole and part of the educational environment with five subscales analyzing the following:

  • Students’ perceptions of learning—12 items, maximum score 48

  • Students’ perceptions of teaching—11 items, maximum score 44

  • Students’ academic self-perceptions—8 items, maximum score 32

  • Students’ perceptions of atmosphere—12 items, maximum score 48

  • Students’ social self-perceptions—7 items, maximum score 28

The total score for all subscales (domains) is 200. Each item is scored from 0 to 4 with scores of 4, 3, 2, 1 and 0 assigned for strongly agree, agree, unsure, disagree and strongly disagree. Items with a mean score >3.5 have a very positive score, and items with a mean score <2 indicate problem areas. Negative items are scored in reverse.

Aim

Using the DREEM inventory we planned to measure the medical education environment in the new curriculum and compare the results with the findings of a previous study carried out in 2004 in the old curriculum [9].

Methods

The DREEM questionnaire was made available electronically to all 394 students enrolled in the new curriculum at the end of the academic year 2008/2009. Students participating were requested to indicate their year of study, gender and nationality. Data were analyzed using the statistical package SPSS 17.0. Analysis of the data included means of total scores, subscale scores and item scores for the whole group and subgroups (male/female, pre-clinical/clinical and Irish/non-Irish). Student t test was used to determine statistically significant differences (p < 0.05). Results were compared with the findings of the 2004 DREEM study on the medical educational environment in the old curriculum.

Results

Two hundred and twenty-five of the three hundred and ninety-four students—57 %—completed the inventory. 65 % were pre-clinical and 35 % clinical; 39 % were male and 61 % female, and 72 % were Irish and 28 % non-Irish.

The total mean score in the new curriculum was 134 out of 200, indicating a more positive than negative score. Compared with the old curriculum (DREEM 2004) this total score was significantly higher—134 versus 130, p = 0.003 (Table 1).

Table 1 Total and domain scores in the new curriculum (DREEM 2009) and the old curriculum (DREEM 2004)
Full size table

Students’ perceptions of learning were significantly higher in the new curriculum, with items in this domain having mean scores equal to or increased in the new curriculum. Four items had statistically significant higher mean scores in the new curriculum compared with the old curriculum. These were the communication skills of the teachers, mean score 2.8 compared with 2.5 in the old curriculum (p = 0.03), opportunities to develop interpersonal skills, mean score 3.1 compared with 2.8 in the old curriculum (p = 0.03), learning about empathy in the medical profession, mean score 2.8 compared with 2.4 in the old curriculum (p = 0.01) and being able to ask the questions they want, 2.7 compared with 2.3 in the old curriculum (p = 0.03). In the new curriculum two items had a mean score of 3.5—“the teachers are knowledgeable” and “I have good friends in the medical school”—indicating real positive points. By comparison none of the DREEM items had a mean score of 3.5 or greater in the old curriculum. Table 2 lists all items with mean scores of 3 or greater in the new curriculum. In the new curriculum four items had a mean score ≤2 indicating problem areas. These were support for stressed students, timetabling, being able to memorize all they need and provision of feedback. Both timetabling and provision of feedback were negatively affected with curricular change while the perception that the teaching over-emphasizes factual knowledge—a problem area in the old curriculum—was no longer one in the new curriculum. The issues of support for stressed students and difficulty memorizing facts were problem areas already identified in the old curriculum and remained problem areas in the new curriculum.

Table 2 Items with mean scores ≥3 in the new curriculum (DREEM 2009)
Full size table

Subgroup analysis: pre-clinical compared to clinical students

In the new curriculum the pre-clinical group had a mean score of 134 and the clinical group a mean score of 136; the difference did not reach statistical significance. Pre-clinical students had higher scores in the subscales of perceptions of atmosphere and students’ social self-perceptions; the difference being statistically significant in the domain of social self-perceptions (p = 0.02). While the clinical subgroup had higher scores in students’ perceptions of teachers and academic self-perceptions these were not statistically significant. There was no difference in perceptions of learning between the two groups.

In the clinical subgroup perceptions of learning, academic self-perceptions and total scores were significantly higher in the new curriculum compared with the old curriculum (Table 3). A number of items had statistically significant higher scores in the new curriculum compared to the old curriculum. In the domain of learning these included encouragement to participate in class, mean score 3.0 compared with 2.6 in the old curriculum (p = 0.002) and “the teaching helps to develop my competence”, mean score 3.3 compared with 3.0 in the old curriculum (p = 0.002). In academic self-perceptions students felt that they were being better prepared for the profession, mean score 3.0 compared with 2.2 in the old curriculum (p < 0.001), they were learning more about empathy, mean score 3.1 compared with 2.7 in the old curriculum (<0.001) and were developing more problem-solving skills, mean score 2.8 compared with 2.4 in the old curriculum (p = 0.001). Inadequate support for stressed students, poor timetabling, difficulty memorizing content and poor feedback from teachers were items identified as problem areas by clinical students in the old curriculum: these items continued to be problem areas in the new curriculum, with no significant difference in scores in the new and old curricula. One item, “I feel too tired to enjoy the course” had a mean score <2 (1.8) in the new curriculum compared with the old curriculum (mean = 2.2): this difference was statistically significant (p = 0.008).

Table 3 Subgroup analysis—clinical compared with pre-clinical students: total and domain mean scores in the new (DREEM 2009) and old curricula (DREEM 2004)
Full size table

Pre-clinical students in the new curriculum had a mean total score of 134 compared with 132 in the old curriculum (Table 3) this difference was not statistically significant. In the domain analysis academic self-perceptions were significantly higher in this subgroup (p = 0.002), with the following items having statistically significant higher scores:

  • I feel I am being well prepared for my profession, mean score 2.8 compared with 2.6 in the old curriculum (p = 0.001)

  • I have learned a lot about empathy, mean score 2.7 compared with 2.2 in the old curriculum (p = 0.001)

  • My problem-solving skills are being well developed, mean score 2.6 compared with 2.2 in the old curriculum (p = 0.001)

Problem areas identified in the new curriculum in the pre-clinical subgroup, i.e., with a mean score <2, were poor timetabling, difficulty memorizing content and poor/inadequate feedback. Pre-clinical students rated provision of feedback more negatively in the new compared with the old curriculum, 1.9 versus 2.5 (p = 0.001).

Subgroup analysis: male compared to female students

Seventy-eight male and one hundred and twenty-one female students completed the inventory, representing 39 and 61 %, respectively. Male students had a total mean score of 137 and females a total mean score of 133; the difference did not reach statistical significance. This is in contrast to the total male and female scores in DREEM 2004 where females had a higher total mean score than their male peers—131 versus 129 (difference not significant). Male students in the new curriculum had a higher mean score in the subscale perceptions of teachers, 31 compared with female students, who had a mean score of 29 (p = 0.02). Male students felt more confident about passing the year, mean score of 3.0 compared with 2.8 in the female subgroup (p = 0.03) and were more comfortable about asking questions, mean score of 3.0 compared with 2.7 in the female subgroup (p = 0.01). Female students, on the other hand, were more confident in their learning strategies than males, mean score 2.4 compared with 2.1 in the male subgroup (p = 0.04).

Male students had a significantly higher total score in the new curriculum—137 compared with 129 in the old curriculum with a p value of 0.003 (Table 4). Perceptions of learning, academic self-perceptions and perceptions of atmosphere also had statistically significant higher scores. Male students found the teaching more stimulating, mean score of 2.8 in the new compared with 2.5 in the old curriculum (p = 0.002) and the teaching helped for developing their confidence, mean score of 2.9 in the new compared with 2.6 in the old curriculum (p = 0.02). They also had more opportunities to develop interpersonal skills in the new curriculum, mean score of 3.1 compared with 2.7 in the old curriculum (p = 0.001). Support for stressed students and difficulty in memorizing everything they need to know were problem areas identified by males in the old curriculum—these remained problem areas in the new curriculum with mean scores of 2 and 1.9, respectively. Satisfaction with timetabling deteriorated with a mean score >2 in the old curriculum (2.1) for becoming a problem area (mean 1.7) in the new curriculum, a difference which was statistically significant (p < 0.001).

Table 4 Subgroup analysis—male compared with female students: total and domain mean scores in the new (DREEM 2009) and old curricula (DREEM 2004)
Full size table

Female students had a total mean score of 133 in the new curriculum compared with 131 in the old curriculum (Table 4). In academic self-perceptions the mean score was significantly higher in the new curriculum (p < 0.001). Items in this domain which scored significantly more positively in the new curriculum were:

  • “I feel I am being well prepared for my profession”, mean score 2.8 in the new compared with 2.5 in the old curriculum (p = 0.002)

  • “Last year’s work has been a good preparation for this year’s work”, mean score 2.6 in the new compared with 2.3 in the old curriculum (p = 0.01)

  • “I have learned a lot about empathy in my profession”, mean score 2.8 in the new compared with 2.4 in the old curriculum (p < 0.001)

  • “My problem-solving skills are being well developed here”, mean score 2.6 in the new compared with 2.3 in the old curriculum (p < 0.001)

Similar to male students, female students in the new curriculum continued to feel there was insufficient support for stressed students and had difficulty in memorizing all they needed to know—with mean scores <2 in the new and old curricula (differences not significant). Timetabling and provision of feedback were negatively affected by the transition to the new curriculum, with mean scores falling below 2; these were 1.6 and 1.9, respectively.

Subgroup analysis: Irish compared to non-Irish students

One hundred and sixty-three Irish and sixty-two non-Irish completed the inventory, representing 72 and 28 %, respectively. The overall mean scores in the Irish and non-Irish groups were 135 and 134, respectively; similarly there were no statistically significant differences in the mean domain scores (Table 5). By comparison in DREEM 2004, non-Irish students had statistically significant lower scores in their perceptions of atmosphere and social self-perceptions compared with their Irish peers.

Table 5 Subgroup analysis—Irish compared with non-Irish students: total and domain mean scores in the new (DREEM 2009) and old curricula (DREEM 2004)
Full size table

Irish students had a higher total score in the new curriculum compared with the old curriculum, 135 and 131, respectively (not significant). They were, however, more positive about their learning and their academic self-perceptions and less positive about their social environment, differences which were statistically significant (Table 6). In the new curriculum Irish students felt more confident about passing the year, mean score of 2.9 in the new compared with 2.6 in the old curriculum (p = 0.004), being well prepared for the course, mean score of 2.9 in the new compared with 2.6 in the old curriculum (p = 0.001), learning about empathy, mean score of 2.8 in the new compared with 2.2 in the old curriculum (p = 0.001), and developing their problem-solving skills, mean score of 2.6 in the new compared with 2.2 in the old curriculum (p = 0.001). They did, however, feel more fatigued, mean score of 2.3 in the new compared with 2.1 in the old curriculum (reversed scored, p = 0.03) and rated their social life lower in the new curriculum, mean score of 2.6 in the new compared with 3.1 in the old curriculum (<0.001).

Table 6 Subgroup analysis: (A) Irish subgroup in the new and old curricula (B) Non-Irish subgroup in the new and old curricula
Full size table

Non-Irish students had a non significant increase in the DREEM mean total score in the New Curriculum (Table 6). Their academic self-perceptions were greater in the new curriculum (p ≤ 0.001), as were their social self-perceptions (p = 0.02). They felt they were being better prepared for their profession, mean score of 2.6 in the new compared with 2.3 in the old curriculum (p = 0.03), their problem-solving skills were being more developed, mean score of 2.8 in the new compared with 2.3 in the old curriculum (p = 0.001) and they were learning more about empathy, mean score of 3.0 in the new compared with 2.6 in the old curriculum (p = 0.001). Timetabling was a problem area in the new curriculum with a mean score of 1.4 compared with 2.2 in the old curriculum (p = 0.001). In addition, provision of feedback dropped from a mean of 2.6 in the old curriculum to 1.9 in the new curriculum (p = 0.001). Difficulty in memorizing curricular content remained a concern in this group with a mean score of 1.8 in the new curriculum and 1.7 in the old curriculum.

Discussion

DREEM has been employed to make comparisons between medical schools [10, 11], expected versus actual perceptions [12] and more recently to compare the environment at different points during curricular reform [13, 14]. Our study compares the educational environment before and after the transition from a traditional curriculum to a systems-based curriculum. Our mean total score of 134 in the new curriculum compares favorably with results from other undergraduate medical schools. For example, from results in developing world, an evolving medical school in Sri Lanka had a mean score of 108 [15] and a traditional undergraduate curriculum in a medical school in Saudi Arabia had a score of 102 [16]. In the developed world overall scores were higher: a new medical school at the University of East Anglia had a mean score of 143 in 2005 [12]; studies at both the University of Dundee [10] and Birmingham University [17] reported overall mean scores of 139 and more recently Lund University in Sweden reported a mean total score of 145 [13].

The key findings in this study that students perceived the overall educational environment and specifically the learning environment more satisfying in the new curriculum are reassuring. This is particularly so when it is acknowledged that the process of change and the rate of change are unavoidably stressful for both students and staff [18]. Furthermore it is encouraging to find that students in the new curriculum agreed more strongly that their problem-solving skills were being developed, that they were learning about empathy and that they had opportunities to develop interpersonal skills. It is also reassuring that the new curriculum has helped to address the more negative perceptions of the atmosphere and their social well-being held by our non-Irish subgroup in the old curriculum. Most of our non-Irish students have less immediate family support and indeed, many do not have English as their first language. In the study on the old curriculum, 97.5 % of the non-Irish subgroup did not have English as their first language compared with 85.5 % in new curriculum. Higher mean scores were found in the following items: “I have good friends”, mean score of 3.3 in the new versus 3.1 in the old curriculum (p = 0.03), and “I seldom feel lonely”, mean score of 2.6 in the new versus 2.3 in the old curriculum (p = 0.02). They felt more comfortable in class socially, 3.0 versus 2.7 (p = 0.02) and disagreed more with the statement “I find the experience disappointing”, mean score 3.0 versus 2.8 (p = 0.04). The change to an interactive curriculum, with more opportunities to participate and small-group teaching sessions, has presumably contributed to their increased satisfaction with the educational environment. It is of interest to note that male students are more positive about the environment than females in the new curriculum. Out of the 50 items in DREEM the male subgroup had statistically significant higher mean scores in 14 items and a lower mean score in 1 item in the new compared with the old curriculum. Females by comparison had statistically significant higher scores in 10 items and lower in 5 items. Females perceived themselves to be more tired, the teachers more authoritarian, timetabling to be a greater problem, to have a poorer social life and receive less feedback in the new curriculum. By comparison males felt only that timetabling had deteriorated in the new curriculum. The reasons why females had a lower increase in their perceptions of the environment after introduction of the new curriculum are unclear at present. This finding should be noted and monitored in future measurements of the educational environment.

Support for stressed students and inadequate provision of feedback remain issues of concern in the new curriculum. Other studies have also found these to be perceived poorly by students, implying these are areas difficult to improve [10, 11, 19]. Effective feedback can be described as feedback in which “information about previous performance is used to promote positive and desirable development” [20]. Giving feedback is frequently a balance between giving honest and accurate information to a student on his/her performance and the student’s ability to translate this into learning and improved performance. It can be compared to walking a tightrope and, at times, can lead to avoidance and negative behavior rather than the desired outcome. This may lead to insufficient feedback being given to students. Congested timetabling and content overload have emerged as new problem areas and will require remedial curricular adjustments. Some of these issues have already been addressed by our curriculum review committee. These include the introduction of a mentoring programme, display of flyers on student supports, introduction of end of module individual feedback sessions and formative assessments at the end of semesters/modules, and student representation on curricular committees. Further investigation and action is required including focus group discussions and communication of key findings to all contributing parties. Follow-up measurements of the educational environment are recommended to ensure these problem areas are responding positively to remedial interventions.

Conclusions

DREEM provides valuable feedback to Medical Schools on the changes in the educational environment following the introduction of new curricula. By measuring the environment appropriate interventions can be made to create a more positive educational environment for all students.