The teaching of critical thinking (CT) skills has been identified as a core area of instruction (Association of American Colleges and Universities 2005; Australian Council for Educational Research 2002; Higher Education Quality Council 1996), because it endows students with the capability to reason not only academically, but also in social and interpersonal contexts where adequate problem-solving and decision-making are necessary on a daily basis (Ku 2009). Though debate is ongoing over the definition of CT and the core skills necessary to think critically (e.g. Bensley 1998; Dewey 1910; 1933; Dwyer et al. 2011; Ennis 1987; Glaser 1941; Halpern 2003; Paul 1993), to date, there has been only one definition and list of skills that stands out as a reasonable consensus conceptualisation of CT. In 1988, a committee of 46 experts in the field of CT gathered to discuss a definition of CT. Ultimately, the group of experts defined CT as:

“…purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based.” (p. 3).

Critical thinking: Skills, dispositions and metacognitive processes

The findings taken from this meeting, known as The Delphi report (Facione 1990), indicated that analysis, evaluation and inference were the core skills necessary for CT (see Table 1 for the description of each skill). According to the Delphi Report, analysis is a CT skill that is used in the context of argumentation to detect, examine and identify the propositions within an argument and the role they play (e.g. the main conclusion, the premises and reasons provided to support the conclusion, objections to the conclusion and inferential relationships among propositions). Notably, at the core of the Delphi definition of analysis is the ability of an individual to analyse the structure of an argument, which depends not only on their knowledge and skill as a reader/listener, but also on the way in which the author of the argument uses relational cues, or signals, that guide the reader/listener (Meyer et al. 1980). For example, words like but, because and however can be used by the author to indicate that propositions that follow are objections, reasons, or rebuttals for propositions that have come before. Another important aspect of analysis is the ability to identify what types, or sources, of propositions others use within their arguments when trying to persuade readers/listeners to share their point of view (e.g. personal experience, anecdotal evidence, common belief, expert opinion, research data, etc.). This form of analysis is important because the ability to identify a proposition’s source allows for a stronger evaluation of the argument.

Table 1 Core CT skills according to the Delphi report (adapted from Facione 1990)
Full size table

Evaluation refers to a CT skill used to assess the credibility, relevance, logical strength and the balance of the propositions and claims within in an argument; thus deciding the overall strength or weakness of the argument (Facione 1990). Evaluating the credibility of claims and arguments involves progressing beyond merely identifying the source of propositions in an argument, to actually examining the credibility of those identified sources (e.g. personal experiences, common beliefs/opinions, expert/authority opinion and scientific evidence). Evaluation also implies deep consideration of the relevance of claims within an argument, which is accomplished by assessing the pertinence or applicability of one proposition to another. Evaluating the logical strength of an argument is accomplished by monitoring both the logical relationships amongst propositions and the claims they infer. Finally, evaluating the balance of an argument involves questioning the motives behind the balance (or imbalance) of supporting and refuting propositions, such as potential bias and/or potential omission of information within an argument.

The final core CT skill, inference, involves the “gathering” of credible, relevant and logical evidence based on the previous analysis and evaluation of available evidence, for the purposes of “drawing a reasonable conclusion” (Facione 1990, p. 9). Drawing a conclusion may imply accepting a conclusion pointed to by an author in light of the evidence they present, or “conjecturing an alternative”, equally logical, conclusion or argument based on the available evidence (Facione 1990, p. 9). According to the Delphi definition, another important aspect of inference is “querying the evidence” available, for example, by recognising the need for additional information or justification and by being able to gather such additional information or justification to draw a conclusion; and to judge the plausibility of utilising such additional information or justification for purposes of conjecturing an alternative conclusion.

Though the three core CT skills identified in the Delphi report have shed some light on what it is we mean by critical thinking skills, at the same time, it is often acknowledged that these skills require time to develop (Dawson 2008; Halpern 2003; King and Kitchener 1994; Kuhn 1999). Related metacognitive processes (i.e. thinking about thinking) may be needed to support critical thinking skill development. The ability to think about thinking (Flavell 1976; Ku and Ho 2010) and the ability to apply CT skills to a particular problem implies a reflective sensibility and the capacity for reflective judgment (King and Kitchener 1994). According to King and Kitchener (1994), reflective judgment is an individuals’ understanding of the nature, limits, and certainty of knowing and how this can affect how they defend their judgments and reasoning in context.

Reflective judgment is often considered as a component of critical thinking (Baril et al. 1998; Huffman et al. 1991), because it allows one to acknowledge that epistemic assumptions (i.e. assumptions about one’s knowledge) are vital to recognising and judging a situation in which critical thinking may be required (King and Kitchener 1994). Furthermore, reflective judgment involves the ability of an individual to acknowledge that their views might be falsified by additional evidence obtained at a later time (King and Kitchener 1994). In this context, reflective judgment is an application of reasoning to complex issues so that judgments can be made even when it is recognised that some problems cannot be solved with absolute certainty. The ability to acknowledge levels of certainty and uncertainty when engaging in critical thinking is important because sometimes the information a person is presented with (along with that person’s pre-existing knowledge) provides only a limited source of information from which to draw a conclusion. Acknowledging this uncertainty and reflectively judging the information despite this, often lead thinkers to reasonably consider multiple, alternative solutions. Notably, some solutions are deemed better than others based on the organisation, complexity and careful consideration of the propositions within the associated argument. Therefore, it is not only the conclusion one reaches, or the inference one draws, correct or otherwise; but also the manner in which one arrives at the conclusion that is important in reflective judgment. It is also worth noting that, unlike analysis, evaluation and inference, reflective judgment development is not a simple function of age or time, but more so a function of the amount of interaction, or active engagement an individual has in the context of working on ill-structured problems, such that the development of higher levels of reasoning and reflective judgment ability can emerge (Brabeck 1981; Dawson 2008; Dwyer 2011; Fischer and Bidell 2006). These perspectives are supported by Bransford et al. (1999), who recommend a “metacognitive approach to instruction” (p. 18) as one of three core principles that emerge from their review of the literature on human learning. The explicit integration of CT training with discipline-based learning can support metacognitive instruction to enhance student achievement.

It is clear from both the Delphi conceptualisation of CT and the existing literature described above that the acquisition of CT skills and the possession of certain dispositions to use these skills are necessary for sustained application of CT across different learning and decision-making contexts. Researchers in the field of CT have noted that it is insufficient for students to only know how to think critically—they must also want to think critically (Ennis 1996; Halpern 2003; Perkins and Ritchhart 2004). That is, along with the ability to engage in CT skills, “a critical thinker must also have a strong intention to recognise the importance of good thinking and have the initiative to seek better judgment” (Ku 2009, p. 71). In the absence of either CT know-how or a strong and sustained intention to apply CT skills, it is unlikely that CT will be applied well across different learning and decision-making contexts (Valenzuela et al. 2011). In other words, both the ability to use CT skills and possessing the disposition to apply these skills together determine a person’s actual CT performance (Ennis 1998; Facione et al. 2002; Halpern 2003, 2006; Ku and Ho 2010).

Critical thinking in educational settings

Though past research suggests that explicit CT instruction can foster growth in CT ability in students (e.g. Reed and Kromrey 2001; Rimiene 2002; Solon 2007), the design of learning environments that actively engage students in the acquisition of CT skills is currently a key concern in education research. For example, in a survey conducted by the University of Western Australia (2007), it was found that while 92 % of academic staff believed it was important to provide students with opportunities to critically evaluate their own beliefs and perspectives with a view towards changing them, 54 % of students felt that they were not actually provided such opportunities by their lecturers/tutors. Interestingly, though most CT interventions are designed based on academic or expert definitions of CT skills, students are rarely, if ever, asked to influence the description or selection of learning outcomes by describing their perspectives on what constitutes CT. That is, to some extent, students are rarely asked to critically think about CT.

Lloyd and Bahr (2010) examined the qualitative descriptions of CT provided by academics and students. Results indicated that students’ descriptions of CT were largely outcome focused (i.e. focused on problem-solving and decision-making outcomes), whereas academics’ descriptions of CT were more focused on the underlying processes of CT (e.g. analysis, evaluation and inference processes) and CT dispositions (e.g. truth-seeking, openness and analyticity). Effective CT instruction implies that instructors should be aware of these and other potential differences between academics’ and students’ initial conceptualisations and descriptions of CT. Students come to class with prior knowledge that must be addressed if teaching is to be effective. This is also highlighted by (Bransford et al. 1999), who argue that student pre-existing understandings (sometimes inaccurate) must be made visible and built upon (supported by metacognitive approaches) for effective learning. If what students know and believe is not engaged, and if the diversity of perspectives in the classroom is not addressed, learners may fail to grasp new concepts and information that are taught and they may fail to understand how they can coordinate their knowledge with the knowledge of others and apply it to real world problems. If prior knowledge and diverse perspectives are not addressed, students may revert to their preconceptions outside of class and develop an insular and fragmented knowledge that is disconnected from other people and real-world problems.

Notably, given the importance of (1) engaging the identity, intentions and goals of students directly; and (2) actively engaging students in defining learning outcomes (Hake 1998; Hogan 2006; Mayer 2004; Redish et al. 1997), it may be beneficial in instructional settings to provide students with CT activities and exercises that are grounded less in academic-, teacher- or expert-centred definitions of CT, but rather the student-centred definitions of CT that are generated in the context of a wider discussion on the very nature of CT. This strategy of developing CT skills and dispositions may provide educators with a better understanding of the CT skills and dispositions that student’s most value—thus, allowing educators to work to: (1) negotiate, co-create and collaboratively design learning outcomes and a CT teaching strategy with students; and (2) foster and maintain students’ willingness to use CT skills while working to achieve learning outcomes.

Methods of negotiation, co-creation and collaborative design need to be carefully selected, such that they engage students in thinking systematically about the nature of specific CT skills and dispositions, and their interdependencies. In the current research, the collective intelligence modelling tool interactive management (see below) was used to support a group of university students in the development of a structural model describing CT skills, dispositions and their interdependencies. However, before discussing interactive management and its use in the current study, it is worth first discussing the impetus for using such a methodology in educational settings with respect to existing educational perspectives.

The Student–teacher relationship: A negotiation

Though the constructivist approach to education has often been applauded as a beneficial approach to teaching and learning, enhancement of learning via the constructivist approach has been criticised for lacking empirical evidence to indicate that novice learners gain knowledge in a given subject area through ‘learning by doing’ (Mayer 2004; Kirschner et al. 2006). Also, students cannot readily construct or apply knowledge about a subject without some previously acquired information regarding that subject (i.e. novice ‘students’ may not possess the necessary mental framework, or schemas required for constructivist learning; Sweller 1999). Likewise, didactic approaches to education are also limited, as they are often linked with low levels of critical thinking, creative thinking and motivation (Hogan 2006). As a result, a balance between teacher-driven knowledge transmission and student-driven knowledge construction may often be required for optimal learning.

With respect to critical thinking, Vygotsky’s (1978) ideas in relation to a zone of proximal development (ZPD) may offer a perspective that aids in achieving this balance (Dwyer 2011). ZPD is the gap between what a student can learn with help and what she can learn without help. Scaffolding is a term used for the method of bridging the gap in the ZPD (Wood et al. 1976). It begins with passive learning, in which the student starts out as a novice and is guided by someone who is more expert in a specific subject than the student herself (e.g. a teacher helps to develop a student’s fundamental knowledge base and lower-order cognitive skills). Once the student becomes more knowledgeable and cognitively capable, the teacher removes some of the scaffolding supporting problem-solving and the student becomes able to guide their own learning and solve increasingly complex problems. As such, the student can ‘learn by doing’ in a way that is increasingly autonomous from expert guidance, provided enough instruction is initially provided.

Though learning through scaffolding has been described as being similar to an apprenticeship (Rogoff 1998), perhaps a partnership between teacher and learner is a more appropriate description. As in any successful partnership, negotiation is often required. The idea of ‘negotiating the curriculum’ has been proposed as having significant advantages over traditional means of curriculum development (Boomer 1992). According to Boomer (1992), p. 13, “if teachers set out to teach according to a planned curriculum, without engaging the interests of the students, the quality of learning will suffer. Student interest involves student investment and personal commitment. Negotiating the curriculum means deliberately planning to invite students to contribute to, and to modify, the educational program, so that they will have a real investment both in the learning journey and in the outcomes.” (Wood et al. 2002) argue that traditional CT instruction may be limited to the extent that it is often designed to “contain all information necessary for a successful solution in their presentation” (p. 279) and they argue that this does not authentically represent real-life, “ill-structured problems”. Authentic negotiation of learning with students must involve a focus on students defining areas of concern for learning that are often ill-structured initially, but where CT has affordances in supporting an effective, rigorous approach to addressing these concerns (and associated learning outcomes). Though Boomer’s perspective is over 20 years old, its current value should not be underestimated, given the ‘new knowledge economy’ implies the ability to identify a purpose and constructively solve problems in the context of an increasing amount of information that is being generated (Darling-Hammond 2008; Jukes and McCain 2002). As such, perhaps one method of enhancing the learning experience, and more specifically, critical thinking instruction, is that of promoting the negotiation of different critical thinking curricula in different teaching contexts. That is, given that CT is often difficult to define (Dwyer 2011; Ennis 1998; Facione 1990; Halpern 2003; Lloyd and Bahr 2010), perhaps negotiating CT curricula will help promote student understanding of CT and help students achieve the learning outcomes expected of them.

The current study

The current case-study aims to provide a ‘voice’ to students with respect to helping them to facilitate their own CT development. The case study sheds some provisional light on the potential affordances of negotiation, co-creation and collaborative design when thinking about the types of CT skills and dispositions that students value and the way in which they see these skills and dispositions to support one another in a system of interdependencies. It seems reasonable to assume that educators must be able to instruct CT not only by means of teaching the curriculum, for example, via presentation and practice of CT skills based on conceptualisations of CT aligned with expert view, including the Delphi report (Facione 1990), or models proposed Ennis (1998), Halpern (2003), Dwyer et al. (2014), and others; but also consistent with the idea of negotiation, co-creation and collaborative design, it would seem be important to attempt to address how students conceptualise CT and subsequently reinforcing the practice and development of appropriate CT skills and dispositions, and correcting potential misconceptions students may have over what they think or believe constitutes CT—a process which may itself be fundamental to the development of critical thinking and reflective judgment (King and Kitchener 1994). Thus, the current research employed interactive management (IM) to develop a student-centred definition of CT, which allows students to generate, select and structure interdependencies between the most important CT skills and dispositions as defined by their working group. Notably, the depth of the deliberation and qualitative insights revealed using the IM methodology is important and a valuable contribution to the scientific community (Harney et al. 2012), particularly given the need to formulate a grounded understanding of students’ perceptions in relation to the nature of CT skills, dispositions and their interdependencies. As such, although provisional in nature, the current research is an important contribution to an on-going effort to better understand the nature of CT and the system of affordances that facilitate students’ CT skill development.

Method

Participants

Participants were 18 second year undergraduate psychology students (7 males and 11 females; aged between 18 and 40 years), from the National University of Ireland, Galway, who took part in the academic module Thinking, Modelling & Writing.

Materials and measures

Based on John Warfield’s (1994) science of generic design, interactive management (IM)Footnote 1 is a thought and action mapping strategy used to aid groups in developing outcomes that integrate contributions from individuals with diverse views, backgrounds and perspectives. IM was designed to assist groups in dealing with complex issues (see Ackoff 1981; Argyris 1982; Cleveland 1973; Deal and Kennedy 1982; Kemeny 1980; Rittel and Webber 1974; Simon 1960). The theoretical constructs that inform IM draw from both behavioural and cognitive sciences, with a strong basis in general systems thinking (Warfield and Cardenas 1994). IM utilizes a set of methodologies, matched to the phase of group interaction and the requirements of the situation. IM commonly utilises the nominal group technique, idea-writing, interpretive structural modelling and profile creation. The first two methodologies are primarily employed for the purpose of generating ideas that are then structured using one or more of the latter methodologies. The current case study used both NGT and ISM to help third-level students identify, clarify and model a set of core CT skills and CT dispositions they felt were necessary learning outcomes.

Specifically, the nominal group technique (Delbeq et al. 1975) is a method that allows individual ideas to be pooled, and is ideally used when there are high levels of uncertainty during the idea generation phase. NGT involves five steps: (a) presentation of a stimulus question; (b) silent generation of ideas in writing by each participant working alone; (c) presentation of ideas by participants, with recording on flipchart by the facilitator of these ideas and posting of the flipchart paper on walls surrounding the group; (d) serial discussion of the listed ideas by participants for sole purpose of clarifying their meaning; and (e) implementation of a closed voting process in which each participant is asked to select and rank five ideas from the list, with the results compiled and displayed for review by the group. A modified version of the standard NGT method was used in the current case study, with participants initially working to identify elements of CT based on their personal experiences and by generating ideas in response to the question: ‘What are the most important skills and dispositions of good critical thinkers?’

Interpretive structural modelling (ISM; Warfield 1994) is a computer-assisted methodology that helps a group to identify relationships among ideas and to impose structure on those ideas to help manage the complexity of the issue. The five steps of ISM are: (a) identification and clarification of a list of ideas (e.g. using NGT); (b) identification and clarification of a “relational question” for exploring relationships among ideas (e.g. “Does idea A support idea B?’’); (c) development of a structural map by using the relational question to explore connections between pairs of ideas; (d) display and discussion of the map by the group; and (e) amendment to the map by the group, if needed.

In the current case study, given our interest in examining the interdependencies between skills and dispositions of good critical thinkers, we focused on enhancement relations, specifically, by asking the following question: “Does critical thinking component A significantly enhance critical thinking component B?” Using the ISM methodology, the group engaged in discussion about each relational question and a vote was taken to determine the group’s judgment about the relationship. A “yes” vote was entered in the ISM software by the computer operator if a majority of the participants judged that there was a significant relationship between the pair of ideas; otherwise, a “no” vote is entered.

The IM approach carefully delineates content and process roles, assigning to participants responsibility for contributing ideas and to the facilitator responsibility for choosing and implementing selected methodologies for generating, clarifying, structuring, interpreting, and amending ideas. Emphasis is given to balancing behavioural and technical demands of group work (Broome and Chen 1992) while honouring design laws concerning variety, parsimony, and saliency (Ashby 1958; Boulding 1966; Miller 1956). IM has been applied in a variety of situations to accomplish many different goals, including assisting city councils in making budget cuts (Coke and Moore 1981); developing instructional units (Sato 1979); designing a national agenda for pediatric nursing (Feeg 1988); creating computer-based information systems for organizations (Keever 1989); improving the U.S. Department of Defense’s acquisition process (Alberts 1992); promoting world peace (Christakis 1987); improving Tribal governance process in Native American communities (Broome 1995a, 1995b; Broome and Cromer 1991); and training facilitators (Broome and Fulbright 1995).

Importantly, IM involves the mapping of arguments generated by a group of knowledgeable individuals. Given that other, similar mapping strategies (e.g. argument mapping) have been shown to facilitate CT processes (e.g. Butchart et al. 2009; Dwyer 2011; Dwyer et al. 2011, 2012; van Gelder 2000, 2001) and learning in collaborative contexts (Engelmann et al. 2010; Engelmann and Hesse 2010; Hwang et al. 2011; Roth and Roychoudhury 1994), IM was considered a good methodology to use in the current context as it would help to catalyse collective intelligence and critical, systems thinking in the process of generating a model of CT competencies.

Procedure

Prior to the IM session, participants completed the academic module Thinking, Modelling, and Writing. Within the Thinking, Modelling & Writing module, a series of in-class and homework exercises were used to develop critical thinking skills, creativity and writing skills. Students were asked to: think about thinking, the nature of critical and creative thinking; characterize their own thinking styles; translate text-based arguments into graphic representations of the argument structure, working to master argument mapping skills; critically review empirical papers and learn about the limitations of different research designs; engage in stimulating dialogue with peers and with their instructor; learn indispensable rules in the art of writing. The module follows Jean Piaget’s teaching principles: every exercise, every action builds upon the previous in an orderly, cumulative and directional manner; as well as Lev Vygotsky’s teaching principles: students are guided into their “zone of proximal development” as they work to build a new set of critical and creative thinking skills. The core module objectives were to develop: critical and creative thinking skills; skill in the art of dialogue; the skill necessary to critically analyse empirical papers; argument mapping skills and writing skills. Performance was assessed via continuous assessment (i.e. 30 %) using in-class and homework exercises and a 2,000 word end-of-semester essay (i.e. 70 %).

After completing the Thinking, Modelling & Writing module, students were brought together for an IM session where they were asked the following question: What are the most important skills and dispositions of good critical thinkers? In order to facilitate open discussion, chairs were arranged in a semi-circle, such that each student could clearly see, and engage with, each other student. Students initially generated a list of 22 skills and 20 dispositions (see Table 2). These skills and dispositions were pooled, and consistent with NGT (Delbeq et al. 1975), students were presented the stimulus question, studied the initial idea set and formulated their thinking in relation to the skills and dispositions before discussing them in order to clarify their meaning. Subsequently, students voted via the selection and ranking of ten ideas from the collated list (i.e. five skills and five dispositions), and the results of voting were compiled and displayed for review by the group.

Table 2 Master list of key skills and dispositions identified by the students
Full size table

After students had a chance to clarify all ideas and rank order their top five skills and dispositions necessary for CT, top ranked ideas were entered into the Interactive Management (IM) software by a trained IM facilitator (see Table 3). Using the IM software, students were then presented a series of questions describing relations among the five skills and five dispositions, for example, Does disposition/skill X significantly enhance disposition/skill Y? (see Fig. 1). Once the students’ discussion had come to a close, a vote regarding the relationship was taken and was entered into the software. A series of over 40 decisions were needed to complete the matrix structuring work. On completion of this discussion and voting session, a structural map describing relations between skills/dispositions was generated by the IM software and projected on a screen in front of the group. Students then engaged in higher-order discussions in relation to the model generated, after which the session closed and participants were thanked for their contributions. The duration of the session was approximately 120 min.

Table 3 Rank-ordered list of the five most important skills and dispositions for CT
Full size table
Fig. 1
figure 1

Screen from the IM software application showing how students were presented with individual matrix structuring decisions

Full size image

Results

A structural map was generated based on the deliberation and voting that took place during the structuring session (see Fig. 2). The structural model generated by the group is to be read from left to right and the arrows indicate ‘significantly enhances’. For example, reading the structure from left to right, a set of 3 CT skills and dispositions can be seen to significantly enhance a total of seven additional skills and dispositions. Specifically, the dispositions: “the willingness to detach from one’s own beliefs” and “the willingness to listen” and the CT skill of being able “to converse and engage with others to expound personal views and experience” are described as the most fundamental drivers of other CT skills and dispositions. A full discussion and interpretation of results is provided below.

Fig. 2
figure 2

Enhancement structure describing relations between CT skills and dispositions

Full size image

Discussion

Interpretation of results

Based on the initial voting of students, the highest ranked CT skill identified by students was the ability to clearly say what it is you want to say, which is closely related to two of Paul’s (1993) elements of reasoning, specifically, the ability to state the purpose of thinking and the question at issue. According to Paul (1993), CT ability is, in part, a function of one’s ability to take time to clearly state both the purpose of thinking and the question at issue; and subsequently, the ability to express oneself in several ways, in order to clarify the meaning and scope of one’s critical enquiry. As described by students in the current case study, the ability to clearly say what it is you want to say is also congruent with a CT sub-skill identified in the Delphi Report, that is, interpretation. According to the Delphi Report, interpretation refers to the ability to “comprehend and express the meaning or significance of a wide variety of experiences, situations, data, events, judgments, conventions, beliefs, rules, procedures or criteria” (Facione 1990, p. 6). In this context, interpretation is further described as a method of clarifying meaning, or the ability to paraphrase or make explicit the meaning of information (e.g. words, concepts or events), in order remove confusion or ambiguity from the initial description (Facione 1990). Furthermore, consistent with the Delphi description of the process of ‘clarifying meaning’, the ability to clearly say what it is you want to say suggests a capacity similar to comprehension as described by others (see Huitt 2011), which refers to abilities related to summarising, paraphrasing and elaborating. In this context, the rank ordering of ability to clearly say what it is you want to say as of the most important CT skill as voted by students in the current research is not surprising, given the fundamental importance of comprehension and the ability to be clear in one’s thinking and expression, as foundational processes of CT in practice (Dwyer 2011; Dwyer et al. 2014; Huitt 2011).

The second and fifth highest ranked CT skills identified by students: Evaluation of the strengths and weaknesses of an argument; and the ability to draw a conclusion about a topic based on its context and what we know about the topic already are closely aligned with the concepts of Evaluation and Inference, respectively (see Table 1). In this context, when evaluating the strengths of an argument, one must make decisions about the strength of propositions with respect to their credibility, relevance, logical form and the potential for bias. Critical thinkers are likely to refute and remove ‘weak’ propositions from their argument structures and progress in their thinking by inferring a conclusion based on the gathering of only the ‘strong’ propositions provided (Fig. 2).

The third highest skill ranked by students and the most influential in the enhancement structure generated by students (i.e. placed at Stage 1 of the problematique) was the ability to converse and engage with others to expound personal views and experience. Broadly speaking, this skill highlights the fundamental importance of dialogue and social engagement in the CT process, which is consistent with the CT conceptualisation developed by Paul (1993). CT is fundamentally a dialectic process, which involves thinking critically about the ideas, logic and arguments that arise in dialogue with others, and reflecting upon different lines of enquiry. The ability to converse and engage with others is central to the practice and development of CT as it provides thinkers with an opportunity to explain and question their own beliefs and arguments, in light of the beliefs and arguments provided by others involved in the dialogue. Furthermore, given that: (1) CT is most often applied in argument-based settings (Allen et al. 1967); and (2) argumentation is “a verbal and social activity of reason aimed at increasing (or decreasing) the acceptability of a controversial standpoint… by putting forward a constellation of propositions intended to justify (or refute) the standpoint” (van Eemeren et al. 1996, p. 5), it seems reasonable to suggest that an open dialogue is not only helpful, but often necessary for conducting CT.

Notably, the fourth highest skill ranked by students—the ability to logically say what you want to in a concise manner, is similar to the second highest ranked skill—the ability to clearly say what it is you want to say, in that both refer, to a large extent, to skills associated with explanation and clarity. However, this fourth skill is more specific in a number of both subtle and important ways. The ability to speak or write logically and concisely indicates that some level of evaluation and inference has already occurred. That is, the ability to logically say what you want to in a concise manner is likely to emerge after evaluation and inference—a verbal or written product that acts as a synthesis, or explanation—an interpretation which is consistent with the enhancement structure generated by students in the current case study. Notably, the skills associated with evaluation and inference appear at Stages 2 and 3 in the CT enhancement structure, followed by the ability to logically say what you want to in a concise manner at Stage 4. This fourth-ranked skill is also congruent with a secondary, or derivative, CT skill identified in the Delphi Report, explanation. Specifically, the skill of explanation refers to the ability to “state the results of one’s reasoning”, justify that reasoning and present that reasoning in the form of cogent argument (Facione 1990, p. 11); and again, this is largely consistent with what the students in the current research describe as the ability to logically say what you want to in a concise manner.

It is worth noting that the skills identified by students related to clarity, evaluation, explanation and inference are all described as CT skills in the Delphi Report, two of them being described as ‘core’ CT Skills (i.e. evaluation and inference; Facione 1990). It is further worth noting the rather concrete manner in which these skills and dispositions are described by students—substantially less abstract than descriptions of similar skills in the existing academic literature. This perhaps reflects the developmental level of students in relation to their thinking about thinking (Fischer and Bidell 2006), with more concrete descriptions of core CT skills and dispositions preceding more abstract representations of the same skills and dispositions at a later date. Consistent with theories of cognitive development (Dawson 2008; Fischer and Bidell 2006; King and Kitchener 1994), perhaps the ability to provide a more abstract representation of each skill is only attainable after the development of some expertise and experience in CT (Dwyer 2011)—or some abstraction from concrete experience, practice and representation of ongoing CT activity.

Furthermore, students identified the willingness to (1) detach from one’s own beliefs and (2) listen properly (e.g. to the lecturer or while reading) as fundamental (Stage 1) CT dispositions. The ability to detach from one’s own beliefs implies a sensibility and capacity that may be fundamental to reflective judgment—an important component of critical thinking (Brabeck 1981; Dwyer 2011; King and Kitchener 1994), that refers to the understanding of the nature, limits, and certainty of knowing, how this can affect the defence of reasoning in context and the acknowledgement that one’s own views might be falsified by additional evidence obtained at a later time (King and Kitchener 1994). Furthermore, the ability to listen has wide-ranging implications, and it implies the willingness to focus attention on relevant and potentially irrelevant information that is presented in the context of ongoing reflective, critical thinking—it implies a self-regulatory function of thinking similar in many respects to executive function (Banich 2009; Gagne 1985; Moseley et al. 2005). For example, central to executive function is the ability to sustain attention and inhibit distractions, both of which are central to good listening skills; and again, this is consistent with what the students in the current case study describe as the willingness to (1) detach from one’s own beliefs and (2) listen properly (e.g. to the lecturer or while reading).

Finally, results suggest that while students’ overall conceptualisation of CT is consistent with existing frameworks, students’ descriptions were, in part, broader, less abstract and more concrete accounts of particular skills identified in existing frameworks; and were also primarily focused on utility or function rather than ideal principles of action. From the perspective of dynamic skill theory (Fischer 1980; Fischer and Bidell 2006), the view of CT proposed by students reflects a level of skill development associated with more concrete and less abstract-systems thinking. It may take time, practice and deep reflection for abstract-systems conceptualisations of CT skills and dispositions to develop—that is, conceptualisation on par with those developed by more expert groups. Thus, in order to facilitate increasingly complex and integrated views in relation to CT and the system of affordances that support CT in practice, it may be important to offer a type of deep reflection and practice that allows for key abstractions and principles to emerge as derivatives of concrete engagement; and possibly also facilitate a broader set of metacognitive skills, including mindful reflection on the nature of thought as it arises in individual and group problem solving situations (see Hogan et al. 2014).

Limitations and future research

While it can be argued that one limitation of the current case-study was the small sample size, it can be argued that this was appropriate for both an exploratory case study of this nature; and furthermore, notwithstanding the sample size, the depth of the deliberation and qualitative insights revealed in this case-study, using the IM methodology, supported the emergence of an important and a valuable contribution to the scientific community, particularly given the need to initiate a programme of research focused on understanding student perceptions in relation to the nature of CT skill, disposition and their interdependencies. Another potential limitation that requires consideration is that students were presented with a number of readings in relation to thinking as part of their Thinking, Writing and Modelling module, including David Bohm (1990) on dialogue and a summary overview of the Delphi report (Facione 1990). Thus, the students in the current research had knowledge in relation to CT that was unique to their learning experience. Future research in this area could examine collective intelligence models generated before and after instruction on dedicated CT modules, that is, to examine the development of students thinking in relation to CT. However, in the current case study it is reasonable to suggest that, without providing students with CT instruction prior to the IM session, the responses and subsequent enhancement structure may not have been as rich and informative, given students’ potential unfamiliarity with CT.

Finally, consistent with perspectives on student/teacher curriculum negotiation discussed above, it is recommended that a larger-scale IM session should be conducted, across multiple student samples, in order to develop the theoretical saturation needed to develop a comprehensive, student-centered, definition of CT that informs the scope and sequence of a new CT curriculum. Results from this proposed programme of research may reveal: (1) further support for the types of skills and dispositions identified by students in the current case-study; and perhaps (2) additional skills and dispositions or high-level themes and influences that are central to students’ conceptualisations of CT; and (3) a progression from more concrete to more abstract-systems CT conceptualisations as students develop their CT skills and dispositions in a university setting. This line of research may support our recommendation that CT instructors should include students’ voices in the teaching/learning relationship and the development of the teaching agenda, perhaps using conceptualisations elicited in this and future research as a starting point for curriculum negotiation with students. This should be seen by educators as a potential opportunity rather than a threat to current practice, particularly given the similarity between what students describe as important in the current case-study and what CT experts highlight as important.

Notably, given that a large body of research indicates that CT instruction can benefit both CT skill performance (e.g. Butchart et al. 2009; Dwyer 2011; Dwyer et al. 2012) and CT dispositions (Rimiene 2002); and given that research suggests that providing students a ‘voice’ in their education, such as through curriculum negotiation and active learning, can benefit student engagement and increase academic performance (Boomer 1992; Hake 1998; Hogan 2006; Mayer 2004; Redish et al. 1997), we suggest that providing a voice to students during CT instruction may, potentially, further enhance both the learning experience and CT performance. Given such a view, alongside the findings from the current case-study regarding how students conceptualise CT (i.e. in a way that resonates positively with existing frameworks), the current study may have implications for how educators approach curriculum design, infused with active learning and negotiation, for improving students’ CT skills and dispositions. Thus, the understanding obtained through this case-study, along with the recommendations for future research, may assist and empower instructors who wish to further understand how their students conceptualise CT, and how they can work collaboratively with students to analyse, evaluate and synthesise their collective perspectives in effort to facilitate learning and the cultural evolution of approaches to teaching.

Conclusions

The design of learning environments in which to optimally instruct CT and allow students to actively engage CT scenarios is currently a key concern in education research; and it is necessary for initiatives in third-level education to focus on guiding students in the development of their CT ability. However, from a Vygotskian perspective (e.g. Vygotsky 1978; Wood et al. 1976), the partnership between student and educator must be reciprocal. As it stands, students are rarely, if ever, asked to guide their instruction by describing their perspectives on what constitutes CT; and thus, to some extent, are rarely asked to critically think about CT.

In the current research, an exploratory collective intelligence study was conducted in order to gain an understanding of students perspectives regarding the skills and dispositions needed to conduct CT. The collective intelligence methodology, Interactive Management, provided students with the opportunity to not only identify the various sub-skills which constitute CT, but also deliberate, vote and structure inter-relationships between highly valued skills and dispositions. Students identified five CT skills, five CT dispositions and fourteen structural relationships among them. The results of the IM sessions build upon research conducted by Lloyd and Bahr (2010), who found that students often conceptualise CT in terms of learning outcomes, by providing further understanding of the relationships between CT skills and dispositions that students consider important. Most notably, students argued that the willingness to detach from one’s own beliefs, the willingness to listen and the ability to converse and engage with others to expound personal views and experience, were the three most important drivers of all other CT skills and dispositions, which also included: the willingness to question assumptions; recognise uncertainty and the limits of knowledge; the ability to evaluate the strengths and weaknesses of arguments; draw conclusions based on what we know; approach the writing of essays in a systematic way; and the ability to logically say what one wants to say in a clear and concise manner.

Fundamentally, the development of CT in educational contexts may depend on cultural factors and in particular how power regulates interactions and who is making decisions about learning. Boomer (1992, p. 6) asks “Are schools dedicated to the promotion of the child’s power to learn, and ultimately to learn independently of instruction and guidance?” These are not value-neutral considerations and research is ongoing on the influence of power on student-centred pedagogy (Donnelly et al. 2014). In any case, given the international movement towards engaging students’ learning goals and outcomes; actively engaging students in defining learning outcomes; and mapping these learning outcomes and implementing action strategies that are consistent with students’ goals (Hake 1998; Hogan 2006; Mayer 2004; Redish et al. 1997), it becomes important to encourage a student–teacher partnership in which student-centred conceptualisations of CT may be incorporated into instruction, fostered, explored and applied. The methodology employed in this study to metacognitively build on preconceptions of CT may have some affordances in providing both a cognitive and psychological scaffold for instructors and students alike. This strategy of developing CT competencies may also provide educators with a better understanding of the CT skills and dispositions towards thinking that students most value—thus, allowing educators the opportunity to facilitate both students’ CT ability and their willingness to apply CT outside of the classroom context.