In problem-based learning (PBL), the role of a tutor or facilitator is considerably different from what is typically considered as the role of a traditional teacher. As described by Barrows and Tamblyn (1980), the facilitator is a metacognitive coach and mentor who probes students to think more deeply and critically and who models for them the types of questions that they should be asking themselves during problem-solving (Collins et al. 1989; Hmelo-Silver 2004). The facilitator also promotes effective group functioning, monitors students’ learning progress, encourages active contributions from team members and intervenes where necessary (Barrows 1988; Maudsley, 1999; Mayo et al. 1995; Wetzel 1996). Moreover, the facilitator should also take a genuine interest in the students and their learning process and seek to develop a collegial and facilitative relationship with the students that is in line with the participatory, collaborative and reflective practice of PBL (Maudsley 1999; Schmidt and Moust 1995).

The role of the PBL facilitator is therefore a complex one and many studies have sought to investigate the characteristics of effective facilitators. Schmidt and Moust (1995) provide a useful framework of an effective tutor that includes three characteristics they propose to be important for effective facilitation: use of expertise, social congruence and cognitive congruence. These three characteristics are reviewed and discussed in greater detail in the sections below.

Use of expertise

As reviewed by Dolmans et al. (2002), one trend of studies examining characteristics of effective facilitators was the rather extensive research into the differences between content expert and non-content expert tutors. The results were inconclusive with some studies demonstrating that content expertise had a positive impact on student learning outcomes (Davis et al. 1992; Schmidt 1994; Schmidt et al. 1993) while other studies indicating no differences (Des Marchais and Black 1991; Groves et al. 2005; Swanson et al. 1990). As noted by Schmidt (1994), subject matter expertise appears to have a greater influence on students’ learning when students had limited prior knowledge or when the resources provided lacked sufficient scaffolding or structure. Thus one possible reason for the inconclusive findings could be that the studies which demonstrated no effect of subject matter expertise on students’ learning were generally conducted using learning materials that were well-structured or where students had a reasonable amount of prior knowledge and vice versa (Schmidt 1994). Another possible reason for the contradictory findings, as commented by Dolmans et al. (2002), could be due to the concept of ‘subject-matter expertise of the tutor’ having a wide range of definitions, with some studies using very rigorous definitions of content expertise while others using broader definitions based on the academic level of the staff to distinguish between experts and non-experts.

Social congruence

Besides subject matter expertise, studies on students’ perceptions of facilitators have identified other characteristics of effective facilitators. One key characteristic focuses on the facilitator’s social congruence. Social congruence refers to interpersonal qualities such as the ability to communicate informally and empathically with students, and hence being able to create a learning environment that encourages an open exchange of ideas (Schmidt and Moust 1995). A study by Kassab et al. (2006) studying the perceptions of medical students (n = 276) from Bahrain found that while students valued content expertise and facilitative skills of the tutors, in addition to those skills, majority of the narrative comments of tutor characteristics focused on tutors being able to establish good rapport with students, for example by respecting their opinions, understanding their feelings and building good relationships. This is in line with what Kaufman and Holmes (1996) found in their study whereby medical students in that educational context expected the PBL tutor to not only be skillful in group facilitation, able to share relevant clinical expertise and anecdotes, but also to have good interpersonal skills such as being friendly and approachable.

These views were similarly echoed by PBL tutors themselves in another study by Papinczak (2010) who investigated the perceptions of both medical students and their tutors on the ‘ideal’ PBL tutor. Both tutors and students highlighted the importance of student–teacher rapport and the need to develop positive student-tutor working relationships.

A study by Chng et al. (2011) also examined the impact of the social congruence of tutors on students’ learning. They found that the social congruence of tutors had a significant influence on students’ learning process as well as their eventual learning outcomes as measured by a concept recall test at the end of each PBL learning phase.

Cognitive congruence

The term ‘cognitive congruence’ appears to be coined by Cornwall (1979) in his explanation on why peer tutors might be better than faculty experts in helping students master concepts. He suggested that peer tutors are likely to have a cognitive schema or knowledge base that is relatively similar to the students and it is this ‘cognitive congruence’ that would enable them to communicate in language that the students would understand as well as to explain concepts in ways that are easily grasped by them. Moust and Schmidt (1994) developed and validated an instrument to measure tutor characteristics including cognitive congruence. They found that indeed student tutors were rated significantly better than staff tutors in terms of their cognitive congruence. Through further interviews with students, they found that student tutors were deemed to be better able to understand the learning challenges students face in understanding and mastering the subject matter. Similar findings were also reported by Lockspeiser et al. (2008) who found that students valued the cognitive and social congruence of peer tutors in an undergraduate medical programme.

While studies on peer tutors indicate that student tutors generally naturally are cognitively congruent with students, this quality of ‘cognitive congruence’ is important in staff tutors too. Schmidt and Moust (1995) suggested that both subject-matter expertise and social congruence are necessary conditions for cognitive congruence to exist. They defined cognitive congruence as a tutors’ ability to express oneself in the language of students, using the concepts they use, and explaining things in ways that are easily grasped by students. They hypothesized that a tutor who is more socially congruent and able to use subject matter expertise better, would be more cognitively congruent. Using structural equation modelling, they demonstrated that both social congruence and subject expertise influenced cognitive congruence, which in turn influenced tutorial group functioning and this indirectly affected the level of student achievement through an increase in time spent on self-study. Social congruence directly influenced group functioning during the problem-solving process while subject-matter expertise of tutors had a slightly direct positive impact on student achievement. Hence, this study showed that effective tutoring that results in better student achievement requires both content knowledge as well as the ability to interact with students on a personal level and to utilize language that is easily understood by students.

Current gaps in the literature on characteristics of effective facilitators

The studies reviewed above provide useful insight into the three broad characteristics of effective PBL facilitators, namely, use of expertise, social congruence and cognitive congruence. However there remain several gaps in our current understanding of students’ perspectives of what makes a good facilitator. First, although the framework by Schmidt and Moust (1995) is a useful one, it is inconclusive which characteristic (i.e. use of expertise, social congruence or cognitive congruence), from the students’ perspective, has the greatest impact on their perception of the facilitator and on their learning in the PBL classroom. Secondly, most of the studies on students’ perspective of PBL facilitators were conducted on medical students (e.g. Caplow et al. 1997; Das et al. 2002; Dolmans et al. 2006; Groves et al. 2005; Kassab et al., 2006; Papinczak et al. 2009; Steinert 2004; van Wyk and Mclean 2007; Yee et al. 2006). Although there are a few studies exploring students’ perception of PBL in other disciplines, including high school geography (Yeung 2010), accountancy (Stanley and Marsden 2012), tourism-related subjects (Huang 2005; Kivela and Kivela 2005; Zwaal and Otting 2010), engineering (Henry et al. 2012), physiotherapy, computer engineering and psychology (Dahlgren and Dahlgren 2002), these studies focused more on examining students’ experience in the process of PBL and did not look in depth into students’ views of effective facilitators. For example the study by Zwaal and Otting (2010) found that students generally rated the performance of their PBL tutors to be above average while in the study by Kivela and Kivela (2005), an analysis of more than 480 student log comments revealed that students viewed the PBL teacher as playing an important role in encouraging independent learning, critical thinking and providing guidance. However there was no further examination on students’ perspectives of the characteristics of effective facilitators in either study. There is therefore clearly a gap in the current understanding of the perceptions of non-medical students on the characteristics of effective facilitators. As seen from the citations above, PBL is currently implemented not only in medical education but across a wide range of disciplines and levels. It is therefore important to find out to what extent previous studies that were mostly based on medical students may be generalized, as well as to identify specific characteristics of effective facilitators from a more diverse group of students. Thirdly, most of the previous studies examining students’ perceptions of effective facilitators had a rather small sample size of hundred students or less, thus further limiting the generalizability of the findings.

The goal of this study therefore was to explore the perceptions of a very large group of students from diverse disciplines (survey responses from more than 10,000 students and from a range of more than 30 diploma programmes), on what makes a good or poor facilitator, with a specific focus on the following questions:

  • What are the specific qualities that may better inform our understanding of use of expertise, social congruence and cognitive congruence, particularly from the viewpoint of students?

  • Out of the three categories of tutor characteristics, namely social congruence, cognitive congruence or use of expertise, which has the greatest impact on students’ perception of the facilitator?

  • To what extent, if at all, do other variables such as the facilitators’ gender and discipline, influence students’ perception of the facilitator?

In so doing, we sought to address the current gap in terms of the limited studies examining the perceptions of tertiary non-medical students on an effective PBL facilitator and to improve the generalizability of our findings through a much larger sample size.

Methods

Ethical considerations

Ethical approval from the institution’s ethics review committee was received for this research. The data obtained were first de-identified by a neutral third party (administrative staff) who was not participating in this study so that that neither students nor facilitators could be identified by the researchers. As this student survey is part of an institutional academic quality process to provide feedback on the facilitators and the modules each semester, it is compulsory for all students to respond to the survey.

Educational context

This study was conducted in a polytechnic in Singapore which delivers more than 230 different modules each semester from a wide range of more than 30 programmes including diplomas in Biomedical Sciences, Information Technology, Aerospace Engineering, Sports and Exercise Sciences and New Media. Majority of the modules are conducted using a PBL approach where each facilitator facilitates at least one class of about 25 students, once a week, over a period of 15 weeks. At the end of each semester, students are required to submit anonymous feedback via an online student feedback survey. For the academic year and semester that this study was based on (year 2011 semester 1), a 96.8 % student response rate (n = 12,358) providing feedback to all facilitators (n = 1,065) was obtained. The mean age of the cohort of students was 18.7 and the percentages of male and female students were 51.8 and 48.2 % respectively.

Instrument

The student feedback survey was adapted from a validated instrument by Schmidt and Moust (1995). The adapted survey in its exact form has been previously validated within this same educational context using confirmatory factor analysis and was found to be both valid and reliable (Williams et al. 2011). For the purpose of this study, the data for this academic year and semester were used to reconfirm the validity of the instrument. The results revealed that the data fitted the hypothesised model well and that the psychometric characteristics of the instrument were adequate, with RMSEA = 0.047, CFI = 0.984 and GFI = 0.987. The survey consists of ten statements for students to indicate how much they agreed with each statement on a five-point Likert scale ranging from ‘Not true at all’ to ‘Very true for me’. These ten statements provide facilitators with feedback on their facilitation skills with respect to their social congruence, cognitive congruence and use of expertise. There are also two optional open-ended questions for students to indicate areas in which the facilitator is good at and areas in which the facilitator needs to improve in.

Social congruence which reflects how well a facilitator is socially aligned with the students was measured by four items: “I was not afraid to tell the facilitator when I did not understand something”; “The facilitator showed that he/she liked informal contact with us”; “The facilitator appreciated our efforts”, and “The facilitator showed interest in our personal lives”. Secondly, cognitive congruence which indicates the ability of a facilitator to communicate and explain the subject matter using words and concepts easily grasped by students was measured by four items in the questionnaire: “The facilitator asked questions that helped us understand the subject”; “The facilitator interrupted us several times, which disturbed the group discussion on the problem/learning task of the day”; “The facilitator helped us to understand the topics at hand”, and “The facilitator unnecessarily used words or jargon that were difficult for me to understand”. Finally, use of expertise was measured by two items: “The facilitator used his/her content knowledge to help us” and “The facilitator has a lot of content knowledge about this module”.

Analyses

The mean scores of each category of facilitator characteristics (i.e. social congruence, cognitive congruence and use of expertise) (n = 1,065) were obtained. Facilitators (identified only by a random number) were ranked from high to low based on the scores obtained for each category. The top and bottom groups of facilitators for social congruence (n = 12 and n = 45 respectively), cognitive congruence (n = 7 and n = 49 respectively) and use of expertise (n = 5 and n = 42 respectively) were defined based on scores that were two standard deviations above and below the mean score for the respective category. Students’ open-ended feedback for facilitators from these two top and bottom groups from each category was analysed further using inductive data analysis procedure (Ryan and Bernard 2003; Thomas 2006). In this procedure, the two authors independently read through the texts to familiarize themselves with the content and to gain an initial understanding of the ideas within the feedback from students. They then independently performed preliminary identification of common categories and themes from 14.8 % (n = 1,102 statements) of the open-ended feedback. Discussions were then carried out to agree on initial themes and to reconcile any differences in opinions. Following that, the remaining data (n = 6,360 statements) from a total of 7,462 statements were equally distributed between the two authors for further analysis. The identified categories and themes were then cross-checked for comparison, with categories combined when the meanings were similar so as to reduce overlaps and to result in the key nine summary categories that were viewed to be the most important themes with respect to the research objectives. Due to the relatively simple textual nature of the open-ended feedback, the main techniques used to identify themes were by identifying repetitions as well as similarities and differences (e.g. contrasts of comments for facilitators who were rated well by students and for those who were not) as suggested by Ryan and Bernard (2003).

In order to provide insights on the category of facilitator characteristic that had the greatest impact on students’ perception of a facilitator, students’ open-ended comments (n = 2,264 statements) for facilitators whose overall scores (i.e. mean score of the three categories) were in the top and bottom groups were coded based on whether the comments were related to social congruence, cognitive congruence or use of expertise. Within each comment, there could be more than one point relating to any of the three facilitator characteristics. Each of these points was individually coded and counted.

Analysis of variance (ANOVA) was used to test whether there were significant differences in terms of students’ perceptions of male and female facilitators with the dependent variable being the student feedback survey scores for use of expertise, social congruence, cognitive congruence and overall score, and the gender of the facilitator being the factor. ANOVA was also used to investigate if the academic department of the facilitator had a significant influence on students’ perceptions of the facilitators’ use of expertise, social congruence, cognitive congruence and overall student feedback score. Eta-squared values were also generated as a measure of the effect size for these analyses.

Results

Descriptive statistics

The descriptive statistics are shown in Table 1. The mean score for use of expertise (4.33 out of a 5 point scale) was highest amongst the facilitators (n = 1,065) while the mean score for social congruence (3.77) was the lowest of the three tutor characteristics.

Table 1 Mean score (Likert scale 1–5) and corresponding standard deviations (SD) for the three categories of facilitator characteristics (n = 1,065)
Full size table

Table 2 shows the numbers of facilitators in the top and bottom groups for each category of facilitator behaviour, as well as the total number of student comments for the respective groups of facilitators. The numbers of facilitators who were two standard deviations below the mean score were much larger than the numbers who were two standard deviations above the mean score for all categories.

Table 2 Number of facilitators scoring two standard deviations above and below the mean for each category of facilitator characteristics and total number of student comments
Full size table

Themes identified

The inductive data analysis of the written comments from students’ feedback on the top and bottom groups of facilitators resulted in the identification of several themes defining students’ perspectives of social congruence, cognitive congruence and the use of expertise. Table 3 provides a summary of the identified themes for each category, definitions of the themes and examples of statements that created the respective themes.

Table 3 Themes identified from the three categories of facilitator characteristics
Full size table

Social congruence

The key qualities of facilitators with high and low social congruence, as perceived by students are categorized and described below.

Facilitator personality

Students described specific qualities related to their facilitators’ personality by expressing their appreciation of facilitators with agreeable personalities such as those who were kind, caring, humorous, approachable, helpful, flexible, patient, and able to trust students. They also identified negative traits which were the opposite of those above, including being unapproachable, inflexible, too serious and strict.

Relating to students

Facilitators who received high social congruence scores were perceived to be empathetic towards students’ needs and expectations. Students felt that the facilitators were able to relate to them, demonstrated a genuine interest in their personal lives and also showed concern for them when needed. In addition, the facilitators were able to communicate actively with students, making the students feel respected and treated as adults or friends.

On the other spectrum, facilitators with low social congruence were viewed by students to be lacking in empathy, not being able to understand the obstacles or difficulties that students were facing in class and life. Due to their lack of interaction with students, these facilitators were unable to establish good relationships with them.

Professionalism

Students appreciated facilitators who were professional in their course of work as demonstrated by being well-prepared for class, hardworking, responsible and prompt. Students perceived these facilitators to be good examples and role models whom they are able to look up to.

On the other hand, students described the lack of professionalism by facilitators who were late in releasing assignments, decreasing the amount of time students were able to work on the assignments. Students also perceived facilitators as being unprofessional when they allowed their feelings and moods to get the better of them, thus affecting the classroom environment and students’ learning in the process.

Motivating students

Students commented that they were motivated when facilitators acknowledged the efforts they had put in. Students appreciated facilitators who were perceived to be aware of the levels of contributions put in by individual students and were able to award grades fairly. These facilitators were able to inspire excellence in students by actively encouraging and by providing constructive feedback to students.

In contrast, facilitators who were perceived by students to be harsh, demoralizing, unable to give constructive feedback and who frequently compared students to other students were not able to motivate students effectively and tended to have low social congruence scores. These facilitators did not readily acknowledge efforts put in by students and were generally perceived as showing favoritism.

Learning environment

Facilitators who had high social congruence scores were perceived by students to be able to establish a conducive learning environment in class. They were perceived to be able to set clear expectations so that students knew what was expected of them. They were also viewed to be well aware of what was going on amongst the students and how the classroom environment was. They were able to pace their lessons well, manage the class effectively and instill good classroom discipline to ensure smooth delivery of the lesson. Students felt that their lessons were engaging and the classroom environment was comfortable to provide ease of learning with minimal stress.

On the other hand, facilitators with low social congruence scores were perceived by students to lack the qualities mentioned above, and students indicated that their classes were stressful or not conducive for learning.

Cognitive congruence

The qualities of facilitators with high and low cognitive congruence, as perceived by students are categorized and described below.

Scaffolding learning

Students generally indicated that they could understand better when facilitators were able to explain difficult concepts using simple words or by breaking them down into simpler concepts. They appreciated facilitators who were able to scaffold learning by using appropriate examples to illustrate their points or by utilising relevant resources to explain concepts. These facilitators were usually described to be systematic in their teaching approach and were able to elaborate well when explaining. They were also able to provide just-in-time help to clear doubts so that students would be able to proceed further in their quest to solve a complex problem or attain all the learning objectives for the day. When learning involved the acquisition of practical skills, the facilitators were able to demonstrate effectively to the students.

Facilitators having low cognitive congruence scores were lacking in the abilities mentioned above, and appeared not to understand students’ learning needs. They also had limited strategies to support students’ learning, thus preventing them from guiding students effectively.

Another quality of facilitators related to effective scaffolding is questioning skills. Facilitators with high cognitive congruence scores were perceived as being able to raise good questions that were critical to aid the students’ understanding of concepts. In contrast, other facilitators posed unclear or ineffective questions that tend to confuse the students and impede their learning.

Communication skills

Students appreciated facilitators who were able to communicate effectively by providing precise, straightforward explanations and not rushing through explanations. In contrast, facilitators who did not have appropriate speed, tone or volume when speaking, as well as facilitators who spoke with a slang or unfamiliar accent affected students’ comprehension of their explanation and instruction. Long-winded explanations or contradiction in the use of words or instructions led to confusion as well.

Use of expertise

Students’ perceptions of the qualities of facilitators with high and low use of expertise are described below.

Content knowledge and expertise

Students rated facilitators high on their use of expertise when they were able to demonstrate immense theoretical and/or practical knowledge and skills of the subject matter in class. These facilitators were also frequently able to share their work experiences in relevant fields of studies to enrich the students’ learning of the subject matter.

Facilitators who seemed to be unsure of the subject matter or were unable to elaborate on the subject matter tended to have lower scores for their use of expertise.

Stretching students’ learning

Students appreciated facilitators who raised challenging questions to stretch their learning and thinking in class, beyond what was in the immediate learning outcomes of the lessons. This was in contrast to facilitators who did not raise enough questions to stretch students’ learning and whom students perceived as asking questions that were not able to invoke deep or critical thinking.

Coding and quantification of comments

Students’ open-ended comments for facilitators whose overall scores (i.e. mean score of the three categories) were in the top and bottom groups were coded based on whether the points in the comments were related to social congruence, cognitive congruence or use of expertise. There were two categories of comments: one regarding what the facilitator is good at, and one regarding what the facilitator is not good at. The total percentages and numbers of points relating to the three facilitator characteristics for the two categories of open-ended comments are presented in Table 4.

Table 4 Percentages of points commented by students relating to each facilitator characteristic for the top and bottom groups of facilitators based on overall score
Full size table

As indicated in Table 4, counts of points related to each of the three categories (use of expertise, social congruence and cognitive congruence) revealed that facilitators whose student feedback scores were more than or two standard deviations below the overall mean score had 57.7 % negative comments (categories under what the facilitators are not good at) as opposed to 42.2 % positive comments (categorised under what the facilitators are good at). Of the 57.7 % negative comments received, majority of the comments were related to social congruence (29.4 %) and cognitive congruence (26.8 %). In contrast, facilitators in the top group received 93.5 % positive comments as opposed to 6.4 % negative comments. More than half of the positive points were related to the facilitators’ social congruence while 31.6 % of the points were associated with cognitive congruence.

The facilitators from the top and bottom groups came from a range of academic departments as indicated in Table 5.

Table 5 Distribution of facilitators who scored two standard deviations above and below the mean for overall score based on their academic departments
Full size table

ANOVA results

The results of the one-way ANOVA revealed that there were no significant differences in overall survey score as well as for scores for social and cognitive congruence for male and female facilitators (overall score: F(1, 1149) = 0.58, p = 0.445; social congruence: F(1, 1149) = 0.28, p = 0.600; cognitive congruence: F(1, 1149) = 1.14, p = 0.285). There was a statistically significant difference for the scores on use of expertise (F(1, 1149) = 6.404, p = 0.012, eta-squared = 0.006) where the mean score for males was 4.35 (SD = 0.28) and for females 4.31(SD = 0.27). Despite the significant difference for the score on use of expertise, the extremely small effect size indicates that this difference is unlikely to be meaningful or noteworthy. One-way ANOVA also revealed that the overall student survey scores, and the scores for social congruence, cognitive congruence and use of expertise differed significantly as a function of the department the facilitator belonged to (F(9, 1141) = 6.471, p = 0.000, eta-squared = 0.05 for social congruence; F(9, 1141) = 6.688, p = 0.000, eta-squared = 0.05 for cognitive congruence; F(9, 1141) = 5.849, p = 0.000, eta-squared = 0.04 for use of expertise and F(9, 1141) = 6.87, p = 0.000, eta-squared = 0.05 for overall score).

Post-hoc analyses using the Bonferroni test showed that only one department (Information technology) was significantly different from the rest on all items (e.g. M = 3.86, SD = 0.28 for overall score as compared to the institutional mean (M = 3.99, SD = 0.27). Once again the extremely small effect sizes indicate that these differences for all three characteristics are probably not meaningful or practical ones.

Discussion

The objective of this study was to examine the perceptions of tertiary non-medical students on the characteristics of good and poor facilitators, using the framework of social congruence, cognitive congruence and use of expertise as developed by Schmidt and Moust (1995). We also sought to identify which of the three facilitator characteristics were perceived by students to have the greatest impact on their rating of a PBL facilitator. In addition, we also investigated if the variables of facilitators’ gender and discipline influence students’ perception of the facilitator.

In order to investigate the perceptions of tertiary non-medical students on the characteristics of good and poor facilitators, we examined the survey results from a large number of students (n = 12,358) from a diverse student population in a polytechnic in Singapore. As this group of students generally consists of academically average students who are also likely to be less mature than the generally older university medical students, we had expected differences in their perceptions of PBL facilitators as compared to other studies done previously. Interestingly we found that the students had similar expectations of what makes a good or poor PBL facilitator, thus indicating that across different levels and fields of study, students’ expectations relating to tutor or facilitator expertise in PBL remain consistent. In addition, this qualitative study extends what is currently known about students’ perceptions of facilitators’ social and cognitive congruence, as well as their use of expertise through the themes identified. We will discuss the findings for each of the facilitator characteristics in the sections below.

Social congruence

For the 1,065 facilitators evaluated in this study, the mean score for social congruence was the lowest (3.77) out of the three facilitator characteristics. For facilitators achieving the highest student ratings in overall score, majority of the points (51.44 %) in terms of areas the facilitators are good in, as commented by the students, were related to social congruence. On the other hand, for the group of facilitators scoring the lowest student ratings, students most often commented that they need to improve on areas related to social congruence (29.43 %). These qualities related to rapport and interpersonal relationships between the facilitator and students are similar to those mentioned by medical students in other studies (Kassab et al. 2006; Steinert 2004). Another study performed by Chen et al. (2006) also revealed that medical students were “highly satisfied with the performance of their PBL tutors regardless of whether they were experts in the subject matter” (p. 627). One of the reasons cited by Chen et al. for this is the way PBL tutorials are conducted in a small group setting that allows for optimal mutual interactions between the tutor and students. Our findings further highlight quantitatively the importance students place on the interpersonal qualities and skills of the PBL facilitator. Thus with respect to the second research question stated earlier, our findings suggest that out of the three categories of tutor behaviour, social congruence is the one perceived by students to have the greatest impact on their rating of a PBL facilitator.

The study by Chng et al. (2011) further reinforces the importance of social congruence in the PBL classroom. They found that although all three tutor characteristics (namely, social congruence, cognitive congruence and the use of expertise) impacted students’ achievement for the day, only social congruence had a statistically significant impact on students’ learning progress as well. Interestingly though, in another study examining the most influential factor in students’ determination of an effective teacher in a non-PBL context of public affairs education, the researchers found that ‘clear explanations’ was the most important factor identified by the university students (Otani et al. 2012). This factor of clarity in explanations was much more influential than the second factor of ‘creating a positive and risk-free learning environment’. This difference in findings of students’ perception highlights the very different expectations and roles of the PBL facilitator compared to that of a typical lecturer or instructor. Most academic staff is likely to have come from a more traditional education background and may therefore not be able to immediately understand or realise this difference in roles and how it impacts students.

Moreover, as cited by Papinczak (2010) and Emery et al. (2003), views of the student feedback survey being a popularity or a personality contest is common amongst teaching staff across a wide range of educational contexts. It is therefore likely that the construct of social congruence may be viewed by facilitators merely as a personality trait or a popularity index. We propose that the findings in this study do not support such ideas and shed light on what students value in a highly socially congruent facilitator. Such a facilitator is not just someone who has the natural personality of being fun-loving and humorous, although these certainly are plus points. Instead students also value facilitators who are caring, approachable, genuinely interested in the students and respect them as young adults, empathetic to their learning challenges and needs, professional in lesson preparation and conduct, motivating and able to encourage students to excel, and are able to maintain a conducive learning environment with an appropriate balance of discipline and flexibility. While some of these traits may come more naturally to facilitators who are more extroverted by nature, majority of the qualities can be acquired and strengthened with reflection and effort by facilitators who are keen to improve in order to better help students learn effectively.

Cognitive congruence

Many of the students’ comments related to cognitive congruence were associated with effective scaffolding of learning including guiding students in their understanding of concepts as well as questioning skills, and effective communication. These findings are in line with the key role of a PBL facilitator as a form of ‘soft scaffold’ (Saye and Brush 2002) who models for students the types of questions students should be asking themselves and whose actions provide a framework to guide students in their knowledge construction (De Grave et al. 1999).

One point to note is that students’ perceptions of effective tutors in relation to cognitive congruence may not necessarily be what are truly helpful for them. For example, what students in this study perceive as being helpful to their learning e.g. by providing more guidance and explanations as well as just-in-time answers, may in fact be withheld by the facilitator because she deems this as being less helpful to the students as compared to temporarily withholding intervention so as to allow students to be more self-directed in their learning. Similar expectations of PBL students on their tutors have been reported in other studies. For example, Kivela and Kivela (2005) reported that students in a hospitality undergraduate PBL programme indicated that the PBL tutor should provide the “necessary answers and information to facilitate the solving of problems” (p. 461). Das et al. (2002) also found that the medical students who participated in their study frequently commented on the need for the PBL tutor to provide more guidance, answers to all questions and even handouts for the lessons. A minority of medical students in a study by Papinczak et al. (2009) indicated their preference for a facilitator with a more directive approach and who would take more control of the students’ learning processes as well as provide more active guidance or intervention. However, such dependency on the PBL tutor is not in line with a holistic concept of scaffolding. As reviewed by van de Pol et al. (2010), the key characteristics of scaffolding include contingency, fading and transfer of responsibility. Contingency refers to being responsive to students’ learning needs and adapting the support provided accordingly. This may include what students in this study mentioned about facilitators being able to use suitable examples or resources to explain and to provide timely assistance to enable them to proceed with the learning tasks. However what was not evident in students’ perceptions of scaffolding were the other two components of scaffolding—fading which involves the gradual reduction of scaffolding based on the students’ development, and transfer of responsibility which involves actively placing the ownership of learning on the student. Thus while students perceive clear explanations and guiding questions as effective scaffolding, the PBL facilitator will need to take into consideration a much broader definition of scaffolding in order to achieve the key objective of PBL in developing self-directed learners who are able to utilise effective strategies for learning and thinking (Hmelo-Silver and Barrows 2006; Hmelo et al. 1997). The use of scaffolding strategies such as metacognitive questioning, encouraging elaboration, seeking clarifications and application of knowledge (De Grave et al. 1999; Hmelo-Silver and Barrows 2006) may necessarily result in some level of student frustration initially. In such instances, the social congruence of a facilitator would then enable him or her to sense students’ frustrations and provide the necessary encouragement to motivate the students to continue with their learning. Thus in PBL facilitation, there is a balance to be maintained between providing too much or too little structure and direction. This balance is one which requires professional judgement and varies with each PBL group and class according to the students’ learning needs and progress.

Use of expertise

In the case of the facilitator’s use of expertise, this had the highest mean score (4.33) amongst the three facilitator characteristics, indicating that most of the facilitators were perceived by students to be relatively strong in their subject matter knowledge. This was also the facilitator characteristic that was least frequently commented on by the students in the open-ended portion of the survey. In the study by Kassab et al. (2006), they found that the frequency of student comments about effective tutors that were related to good content knowledge made up 22.22 % of the narrative comments, as opposed to a higher frequency of comments related to building of rapport (52.38 %) and helping students (25.40 %). Similar to the findings in our study, the student participants in the study by Kassab et al. (2006) appreciated tutors who were able to provide illustrations from their own clinical experience and who had breadth of knowledge.

The second theme related to the use of the use of expertise identified in this study was “stretching students’ learning” e.g. by raising critical or challenging questions. This aspect of the use of expertise was also reported in a study by Gilkison (2003) examining the differences in techniques used by ‘content expert’ and ‘non-content expert’ tutors in a PBL class. Her analysis of the interactions in the PBL tutorial groups of a subject matter expert and non-expert tutor revealed that the tutor with content expertise made use of intervention techniques that “raised critical awareness” to a much larger extent (48.1 % of tutor interventions) as compared to the non-expert (38.1 %). Interviews with students also indicated that students valued how the tutor with a medical background was able to raise critical awareness through questions that helped them think about issues they would not have otherwise thought about. On the other hand, other studies comparing the tutor–student interactions in PBL groups led by experts and non-experts found that the former tended to be more teacher-directed and these tutors generally took on a more directive role (Davis et al. 1992; Kaufman and Holmes 1998; Silver and Wilkerson 1991). It is therefore important, as pointed out by Kassab et al. (2006) and Caplow et al. (1997), for tutors to not only have content expertise but also facilitative expertise. As discussed by various authors (e.g. Hmelo et al. 1997; Mayo et al. 1995), the goals of PBL go beyond the learning objectives of the lesson and include helping students develop reasoning, critical thinking and self-directed learning skills. Thus an effective PBL facilitator’s ‘use of expertise’ should include both the use of content knowledge expertise as well as facilitative expertise.

Other variables influencing students’ perception of the facilitator

The third objective of this study was to explore if other factors such as the facilitator’s gender or discipline area influenced students’ perception of their effectiveness of a PBL facilitator. Results from this study generally indicate that there is no significant difference in students’ perceptions of male versus female facilitators. Even though for use of expertise, male facilitators had a statistically significantly higher rating, the effect size was extremely small (eta-squared = 0.006), thus negating the usefulness of this difference.

In the case of the departments that the facilitators belonged to, facilitators from one department (Information technology) received statistically significantly lower scores compared to the other eight academic departments. However once again the effect size was very small (eta-squared = 0.05), thus indicating that the difference is not a meaningful one. We would like to highlight the observation that for all the other departments, there were no statistically significant differences in the students’ perceptions of the facilitators. These results are interesting especially since they are from a very large sample size and wide range of academic departments, and they strongly indicate that students’ perceptions of facilitators’ characteristics are not significantly influenced by the area of discipline being taught or the gender of the facilitator, thus reinforcing the generalizability of the findings from this study across a wide range of disciplines.

Implications for staff development

The findings from this study provide input on what a facilitator can do to be more socially and cognitively congruent and to enhance his or her use of expertise. In addition, these findings also have implications for PBL staff development at the institutional level. As educators in a tertiary education setting, most teaching staff would not argue against the need for cognitive congruence or effective use of one’s expertise. However, not all facilitators would be aware of students’ needs for a socially congruent facilitator. Thus we recommend that faculty development programmes in PBL increase staff awareness of students’ perceptions of socially congruent facilitators, and highlight research findings on how social congruence does impact students’ learning process and achievement (Chng et al. 2011; Schmidt and Moust 2000). The authors propose that the findings in this study that describe in detail what students value and suggest for facilitator improvement in the area of social congruence are useful points for facilitators to note and reflect upon.

In the area of cognitive congruence, we have discussed that this requires a delicate balance and ability to monitor and adjust one’s facilitation approach based on the specific students’ learning needs and challenges. This is a skill that on one hand requires experience, and on the other hand, needs ongoing reflection and feedback in order to improve. Staff development programmes should therefore provide opportunities for self and peer review of one’s facilitation, as well as peer-coaching by more experienced and effective facilitators (Orlander et al. 2000). Such support mechanisms are likely to be very useful especially for new PBL facilitators.

Finally, staff development programmes should include specific strategies to help facilitators work towards the larger goals of teaching and learning which include helping students develop reasoning and critical thinking skills, metacognitive thinking and self-directed learning strategies. Opportunities should also be given for facilitators to reflect as well as obtain peer feedback on the impact of their use of expertise (both content knowledge and facilitative expertise) on developing students towards these broader goals of PBL.

Limitations of the study

It is important to note several limitations of this study. First, this study focuses only on students’ perceptions without considering the viewpoints of the facilitators. Further studies to examine the tutor’s perspective of good facilitation in a non-medical education context should be undertaken to understand if there are any significant differences in the perceptions of roles and values. Secondly, due to the focus and nature of this study, the findings cannot demonstrate a relationship between students’ perception of facilitators and their learning outcomes. Supplementing students’ self-report with additional data such as students’ learning achievements could have provided stronger evidence on the impact of these tutor characteristics on students’ learning. Thirdly, it is likely that students’ perspectives on facilitators would depend on their perceptions of the value of the PBL approach. However in this study, we were not able to obtain information on this possible relationship, and how (if at all) either perspective influences the other. Also, although we have demonstrated that students’ perceptions on facilitators were not influenced by the facilitators’ gender or area of discipline, due to the de-identified data we were working with, it was not possible within this study, to examine if the students’ own gender and major of study had an impact on their perceptions. Lastly, in the identification of the category of facilitator behaviour that has the greatest impact of students’ perception of a facilitator, the assumption made was that students would comment most frequently on what matters most to them. Although we propose that this is a reasonable assumption, it is possible that other non-related factors could have resulted in students commenting most frequently on an issue.

Conclusion

The findings from this study provide an insight into students’ viewpoints of what makes a good or poor facilitator and help facilitators and staff developers better understand the specific qualities related to the PBL facilitator’s social congruence, cognitive congruence and use of expertise. In particular, we found that out of the three facilitator characteristics, social congruence appeared to have the largest impact on students. While some may question the reliability and validity of findings obtained from student perception, the authors agree with Lea et al. (2003) on the importance of researching and understanding students’ expectations and perceptions. They argue that in a student-centred learning environment, it is ‘simply good educational practice’ (p. 324) to take into account students’ needs and expectations in the teaching and learning process. Moreover students are likely to be better able to stay on with a course that meets their learning needs. Indeed this second point is particularly important in a PBL environment where students who have been used to a more traditional teacher-centred teaching approach often face a significant amount of initial discomfort and apprehension. Thus in line with the learner-centred philosophy of PBL, it is not only valid but necessary to research from students’ perspective to better understand their learning needs in order to inform staff professional training and development. From the authors’ knowledge, this study is also the first one that examines student survey responses obtained from more than 10,000 students.

Having argued for the validity of this study, we recognize that further studies to examine the tutor’s perspective of good facilitation in a similarly broader tertiary non-medical education context should be undertaken to understand the other perspective and to identify if there are any significant differences in the perceptions of roles and values. Similarly additional studies to investigate how these facilitator characteristics directly impact students’ learning outcomes (or not) and the extent to which students’ perceptions of PBL influences their perception of PBL facilitation should also be examined further. These research efforts are important to provide a balanced view to inform and improve our practice of the complex art of PBL facilitation.