Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

Web 2.0 technologies can be used in education, especially for building project-based learning activities (Duffy & Kirkley, 2004). Among these technologies, wikis seem to offer rich collaboration possibilities (West & West, 2009). A wiki offers the ability to edit a website by adding, modifying, and deleting pages as well as integrating hypermedia.

The open nature of the wiki technology provides opportunities for learning (Mindel & Verma, 2006; Raman, Ryan, & Olfman, 2005; Wheeler & Wheeler, 2009), since all participants should work collaboratively in order to edit and improve the content. Wikis as a collaboration tool can help students to write better (Mak & Coniam, 2008) and can support collaborative knowledge creation (Raman et al., 2005; Wagner, 2004). In addition, wikis can facilitate group learning (Carpenter & Roberts, 2007), foster contribution to peers (West & West, 2009), and improve students’ engagement (Molyneaux & Brumley, 2007).

Various skills, such as writing, IT, collaboration, and organizational skills (Lai & Ng, 2011; Wheeler & Wheeler, 2009), can be improved with students’ involvement in wiki-based activities. Thus, students’ participation in wiki projects can add value to learning and to their professional success (Frydenberg, 2008). It seems that in well-designed wiki-based activities students are positive towards this technology (Tétard, Patokorpi, & Packalén, 2009). They believe that wikis are useful for sharing knowledge (Elgort, Smith, & Toland, 2008) and a great tool for collaboration (Deters, Cuthrell, & Stapleton, 2010). Unsurprisingly, wikis have been used in various contexts such as developing an online textbook (Ravid, Kalman, & Rafaeli, 2008) and supporting knowledge sharing (Raman et al., 2005). Wikis can also help teachers to manage and mark their students’ work (Deters et al., 2010). However, the wiki openness may also be a disadvantage if the context and objectives of the activity are not well determined (Parker & Chao, 2007). As a result, there are examples in which students did not actively participate in creating or editing context (Cole, 2009; Ebner, Kickmeier-Rust, & Holzinger, 2008).

In order to avoid these situations and to provide a rich context and simultaneously a support structure in a wiki environment, West and West (2009) proposed an instructional design process. This process includes the following steps: establish a purpose for the wiki project, define and classify the wiki project’s learning goals, design a rich context and problem that support the achievement of the purpose and goals, prepare students for work in the new environment, and promote a collaborative process through which active, social learning can take place (West & West, 2009, p. 22). This approach emphasizes scaffolding. In addition, West and West (2009) used the Bloom’s Taxonomy of Learning (Bloom, 1956) to classify the main wiki project’s learning domains. From this grouping three wiki project categories emerged: (a) knowledge construction, (b) critical thinking, and (c) contextual application. The wiki activity design was based on the framework proposed by West and West (2009), and its learning domain belongs to the category of knowledge construction. The goal of the designed activity was to learn general information about Web 2.0 and its applications in the frame of a first-year academic course entitled “Introduction to ICT.”

The aim of the study presented in this chapter was to investigate the effect of a framed, rigorously designed, wiki-based activity on the learning outcome. In specific, this chapter investigates:

  • Students’ learning gain after the wiki-based activity

  • Whether the students with lower pretest score benefited from the activity at least to the same extent as students with higher pretest score

  • Whether students’ learning performance was affected by their role while carrying out the activity

  • Whether students with more logged wiki edits benefited more than students with less wiki edits

This chapter is organized as follows: Initially, the research methodology, the profile of the participants, and the design of the activity are described. Subsequently, the research results are presented, focusing on learning outcome as assessed by an appropriately designed pre- and posttest questionnaire.

Methodology

Research Method and Materials

A single-group pretest–posttest design was adopted (Cohen, Manion, & Morrison, 2000). A questionnaire with closed questions was the data collection instrument. The pretest questionnaire comprised both demographic (13 questions) and factual knowledge questions (36 questions, each with four possible answers of which only one was correct). The factual knowledge questions were primarily related to general information about Web 2.0 and its applications, whereas the demographic questions were related to personal information regarding ICT, Internet, and wiki usage and adoption. The posttest comprised the same factual knowledge questions. The students were not informed that they would be asked to complete the questionnaire at the beginning or the end of the activity.

The wikispaces service (www.wikispaces.com) was used both for the activity announcement and as the platform provided to the students to construct their wiki. The online questionnaire service SurveyMonkey (www.surveymonkey.com) was used to create and distribute the questionnaires of the study. The obtained data were organized and analyzed using Excel 2007 and SPSS v17.0. The activity presentation, students’ presentation of their wikis, and completion of questionnaires took place in the computer lab of the Department.

Procedure and Participants

All in all, 220 first-year university students participated in the study. The students were divided freely into 44 groups comprising 5 members each. In the beginning of the procedure an instruction on the wiki’s basic functionality was given to the students. Subsequently, a compulsory assignment was presented to them in the form of a wiki, realized by the researchers. Each team member had a specific role in the group such as collector, organizer, editor, and verifier (West & West, 2009). The responsibilities of each role are delineated in the following.

Eighty-one (81) of the students did not respond to either the pre- or posttest assessment questionnaire and were excluded from the dataset. Analysis was conducted for the data collected by 139 students, 2 male and 137 female, aged 17–37 (mean = 19.3, sd = 3.5). The majority of the participants (127/139) were 17–22 years old. They were attending a compulsory academic course entitled “Introduction to ICT,” offered in the first semester in the Department of Education and Early Childhood Education at the University of Patras. Participation in the activity was compulsory and was one of the five required mini-projects given to the students in the context of the laboratory part of the lesson.

Description of the Activity

The design of the activity was based on the framework proposed by West and West (2009). Learning was expected to be achieved by engaging the students into four processes: information seeking and retrieval, argumentation development and refinement to support their thesis, cooperation among members, and their involvement with the wiki-editing process. The assignment was presented to the students through an exemplary wiki, which was constructed by the researchers (available at http://labtpewiki.wikispaces.com).

The exemplary wiki included the purpose and the objectives of the assignment, detailed implementation instructions, expected learning outcome, evaluation criteria, and representative support material. In addition, the topics that students had to cover were outlined and organized into subsections with a short description for each one. Afterwards, the students of each group had to create their own wiki, in which they would develop the topics of the assignment.

The topic of the designed activity was to learn general information about Web 2.0 and its applications. It was selected due to the following reasons: First, the students should be able to understand the impact of Web 2.0 on society in general and on education in particular. In addition, the topic is suitable for covering a variety of educational and technological aspects of Web 2.0, thus giving a fertile ground for argumentation. Finally, it is a notable session of the course’s overall outline. The exemplary wiki included nine segment topics: (a) Web 2.0 definitions, (b) characteristics of Web 2.0, (c) YouTube and Slideshare, (d) Twitter, (e) Blogs, (f) Wikis, (g) Skype, (h) description of an educational activity using these technologies, and (i) potential risks of Web 2.0 use.

For this activity wikis were used to promote collaboration with peers and improve their knowledge related to the aforementioned topics (West & West, 2009). The students had to search for information on all of these topics and seek additional material. Furthermore, it was stressed that usage of other’s work should follow specific rules since the open nature of Web 2.0 tools could lead to inappropriate use of content from other sources, as reported in Huijser (2008) and West and West (2009). Students were instructed on how to use and cite sources and were also informed that they could only use freely available media or media under a creative commons license.

Each team member was assigned a specific role by the researchers. These roles are delineated by West and West (2009). The first role was that of “collector” who had the responsibility to obtain appropriate material relevant to each subtopic. Two members of each group were “collectors.” The second role was that of the “organizer” who was responsible to organize the collected material and to check its consistency and relatedness with the objectives of the project. The “editor” was responsible to check grammar and syntax errors in the content and its compliance with the provided format. The “verifier” was responsible to check the content for its completeness, structure, and compliance with the objectives of the project. However, all students were allowed to participate and contribute in every aspect of the collaboration process.

Finally, the students had to present their work briefly during the laboratory session of the course. Each project was graded by the researchers on a 1–100 scale. The score was multiplied by the number of the group members and was given to the students. Subsequently, the students in each group were asked to discuss and distribute these points fairly according to each member’s contribution. As far as the score distribution is concerned, a notable differentiation was observed in only 9 out of the 44 groups. Such grade distribution differentiations possibly indicate a lack of balanced collaboration.

Results

All in all, we analyzed data from 139 first-year university students involved in a wiki-based activity in the context of an introductory ICT course. Table 1 presents participants’ demographic-related information in our dataset.

Table 1 Participants’ demographic-related information in our dataset
Full size table

First, a reliability analysis of the provided 36-item knowledge assessment questionnaire was conducted. Reliability refers to the extent to which an instrument, such as a questionnaire, yields the same results under consistent conditions (Nunnally & Bernstein, 1994). It is most commonly measured using Cronbach’s alpha, which is a measure of internal consistency. Results showed that the initial 36-item questionnaire used in the study did not have sufficient reliability (alpha = 0.69) to meet the typical minimum standard of 0.70 (Nunnally & Bernstein, 1994). Two questions increased the alpha to 0.70 if they were deleted and thus were excluded from subsequent test score computations.

Next, students’ pretest and posttest correct answers in the knowledge assessment questionnaire were converted to a composite test score on a 0–100 scale. In addition, a normalized learning gain score was produced for each participant by using the formula proposed in Nelson et al. (2009) and defined as

$$ G=\frac{{\mathrm{post}}_{\mathrm{score}}-{\mathrm{pre}}_{\mathrm{score}}}{{ \max}_{\mathrm{score}}-{\mathrm{pre}}_{\mathrm{score}}} $$

This score has the advantage of “normalizing the observed gain (the numerator) against the amount of possible learning that could be achieved (the denominator)” (Nelson et al., 2009, p. 1797). Table 2 presents descriptive statistics of the dependent variables measured in the study.

Table 2 Students’ pre- and posttest scores in the knowledge assessment questionnaire and their normalized learning gain
Full size table

In all subsequent statistical analyses, we use the correlation coefficient r as an effect size, which is calculated according to the formulas reported in Field (2009).

Did the Wiki-Based Activity Improve Students’ Performance?

A dependent t-test was applied to compare students’ pretest (M = 43.6, SD = 11.4) and posttest (M = 63.8, SD = 13.6) performance, as measured by the provided knowledge assessment questionnaire. The differences between the test scores did not violate the assumption of normality (D(139) = 0.98, p = 0.059), and thus a parametric test was selected. Results indicated that students achieved significantly higher (t(138) = 17.74, p < 0.001, r = 0.83) test scores after participating in the wiki-mediated learning activity. According to Cohen (1992), this is a very large effect size, which demonstrates the learning effectiveness of a properly designed wiki-mediated learning activity.

Were Students with Lower Pretest Score Benefited at Least to the Same Extent as Students with Higher Pretest Scores?

We recoded our dataset to create two between-subject groups based on students’ initial performance: (a) low initial performance (N = 83), which included students with pretest score below or equal to the median score of all students, and (b) high initial performance (N = 56), in which students with pretest score above the median score of all students were assigned. Table 3 presents descriptive statistics of students’ pretest score, posttest score, and normalized learning gain in relation to these groups.

Table 3 Students’ pre- and posttest scores in the knowledge assessment questionnaire and their normalized learning gain grouped by their initial performance
Full size table

A two-tailed Mann–Whitney U test investigated the effect of students’ initial performance on their normalized learning gain. A nonparametric test was selected because the assumption of normality was violated for the high initial performance group (D(56) = 0.95, p < 0.05), and homogeneity of variance was also violated (Levene’s test, F(1,137) = 9.39, p < 0.01). Results indicated that although students with low initial performance showed a slightly higher (3.9 %) normalized learning gain compared to those with high initial performance, this difference was not significant (z = 0.14, p = 0.889). Also, an insignificant correlation (r s = −0.04, p = 0.668) between students’ pretest score and normalized learning gain was found.

Additional analyses after Bonferroni correction investigated whether the wiki-mediated activity improved students’ score for both the low and high initial performance groups. A nonparametric test was selected for the high initial performance group because the distribution of the differences in the dependent variable (test score) between the two related conditions deviated significantly (D(56) = 0.94, p < 0.01) from a normal distribution. Results showed that students’ test scores were significantly improved in both the low and high initial performance groups: t(82) = 17.16, p < 0.001, r = 0.88 and z = 5.56, p < 0.001, r = 0.53, respectively.

All in all, the above results provide evidence that the wiki-mediated learning activity was beneficial to students with lower initial performance, at least to the same extent as those with higher initial performance.

Did Students’ Role in the Wiki-Mediated Activity Affect Their Learning Gain?

Table 4 presents students’ performance grouped by their role in the wiki-based activity: collector (N = 53), organizer (N = 28), editor (N = 29), and verifier (N = 29).

Table 4 Students’ pre- and posttest scores in the knowledge assessment questionnaire and their normalized learning gain grouped by their role in the activity
Full size table

Table 4 shows that the lowest average learning gain (30.0 %) was observed for students with the verifier role, whereas students with the editor role had the highest learning gain (44.4 %) on average. However, a one-way ANOVA did not unveil any significant learning gain differences between the four different students’ roles (F(3,135) = 2.48, p = 0.064).

Thus, results show that the wiki-mediated learning activity was beneficial to all students, regardless of their specific role in the project.

Were Students with More Wiki Edits Benefited More Than Students with Less Wiki Edits?

We recoded our dataset to create two between-subject groups based on students’ logged number of edits in the wiki: (a) low number of wiki edits (N = 71), which included students with a number of wiki edits below or equal to the median number of wiki edits of all students, and (b) high number of wiki edits (N = 68), in which students with a number of wiki edits above the median number of wiki edits of all students were assigned. Table 5 presents descriptive statistics of the measured dependent variables in relation to these two groups.

Table 5 Students’ pre- and posttest scores in the knowledge assessment questionnaire and their normalized learning gain grouped by their number of wiki edits
Full size table

The assumption of normality was violated for the high number of edits group (D(68) = 0.93, p < 0.001); thus a nonparametric test was applied to investigate differences between the two groups. A two-tailed Mann–Whitney U test showed that students with more wiki edits had a significantly higher learning gain (z = 4.32, p < 0.001, r = 0.37) compared to students with less wiki edits. In addition, a significant correlation (r s = 0.41, p < 0.01) was found between students’ number of logged wiki edits and normalized learning gain: the more active the students were, the more they improved their performance.

Additional analyses after Bonferroni correction investigated whether the wiki-mediated activity improved students’ score for both the students with lower and higher number of wiki edits. Results showed that students’ test scores were significantly improved in both groups: t(70) = 10.69, p < 0.001, r = 0.79 and t(67) = 15.45, p < 0.001, r = 0.88, respectively. As the effect sizes show, this improvement was of higher magnitude for the students who were more active contributors in the wiki.

In sum, it was found that both students with low and high number of logged edits in the wiki improved significantly their performance, but the latter had a significantly higher learning gain.

Conclusions

A study investigating the effectiveness of a wiki-mediated learning activity for ICT education was presented. The evaluation was carried out using a one-group pretest–posttest design. The results showed significant improvement in learning outcomes; the average students’ test score improved from 43.6/100 to 63.8/100. In addition, it was found that the wiki-mediated learning activity was equally beneficial to students with lower and higher initial performance. These results suggest that a properly designed, framed wiki-based activity could substantially facilitate students to learn by building content. In a similar vein, there are surveys which indicate that wiki technology can be beneficial to students in various learning domains (Mohammed, 2010), given that a carefully designed activity is introduced to them. Similar findings are reported in Tselios, Altanopoulou, and Katsanos (2011) and Tselios, Altanopoulou, and Komis (2011).

No significant learning gain differences between the four different student’s roles (i.e., collector, organizer, editor, verifier) in the wiki activity were identified. This finding is in line with previous research (Strijbos, Martens, Jochems, & Broers, 2004; Tselios, Altanopoulou, & Katsanos, 2011; Tselios, Altanopoulou, & Komis, 2011) which indicates that roles do not affect group members’ performance. However, roles can help students who work collaboratively to build knowledge in comparison to students with no distinct roles while collaborating (Schellens, Van Keer, De Wever, & Valcke, 2007).

Furthermore, results showed that students who were more active contributors in the wiki, as measured by their logged number of edits, had a significantly higher learning gain.

However, the reported study is not without limitations. First, it should be noted that our data is gender and age skewed; thus the findings might not be generalizable to male students or older students involved in wiki-mediated learning activities. Furthermore, the results obtained do not explain how the students have benefited from their involvement in the activity. Future research goals constitute the design of additional wiki-based activities in a variety of educational settings as well as investigation of the learners’ behavioral intention to use wiki technology using technology acceptance models (Tselios, Daskalakis, & Papadopoulou, 2011). Moreover, the relation between the observed students’ activity and the learning outcome will also be examined (Katsanos, Tselios, & Avouris, 2010).