Introduction

The prevailing Web 2.0 applications share many merits in supporting pedagogical goals such as participation, engagement, discussion and collaboration (Grosseck 2009). The usefulness of applying new technological applications such as podcasting to engineering education has received increasing interest in the literature (Palmer and Hall 2008). Researchers and educators in the field of engineering education have paid increasing attention to the value and application of information communication technology (ICT) to online learning and instruction in engineering (Bourne et al. 2005). The employment of computing and communication technologies has been found to have potential value for engineering-related courses and laboratory activities with regard to assisting engineering students in developing critical competencies necessary for life-long learning (Balamuralithara and Woods 2009; Carroll et al. 2007; Fang et al. 2008). In addition to the delivery of engineering-related course content and learning systems, engineering students are expected to exercise practical skills and construct engineering knowledge in ICT-supported learning environments. For example, through employing an online discussion forum to support an engineering management course, Palmer et al. (2008) indicated that engineering students’ preparation in online discussions was helpful to their online communication skills for task completion and course performance.

Within the social Web 2.0-based context, the transformation in learners’ participatory and creative practices may alter what and how learning occurs (Huang et al. 2009). In such a context, learning may require users to represent, share and communicate their experiences, ideas and opinions with others for knowledge construction in social networking sites, a process which emphasizes student centeredness, peer negotiation, knowledge construction and co-construction (e.g., Jonassen et al. 2003; Tsai 2001). The main concepts and features of many Web 2.0 applications mostly concur with the constructivist pedagogy which encourages learners to construct personal understandings in socially interactive environments. Furthermore, conducting Web 2.0 applications in educational practice features knowledge construction as decentralized, accessible, and co-constructed activities through peer review in an engaged community of users (Greenhow et al. 2009). Such new learning environments may provide opportunities for learners to exercise inquiry-oriented activities of gleaning data and interpreting the data to answer their own questions, thus facilitating their engagement in and development of critical thinking and high-order learning. In this regard, research on students’ Web 2.0 activity in terms of their participation, investment and knowledge building may provide researchers and educators with more clues to the potential of different Web 2.0 applications for academic purposes.

Online collective information searching (CIS) activities

The advent of Web 2.0 applications has been deemed as a potential means of supporting learning and teaching (Huang et al. 2009), and is gradually altering the ways in which we conventionally access the Internet from passive one-way information retrieval to active two-way information creation and communication (Mendenhall and Johnson 2010). Collective information searching (CIS) activities, supported by Web 2.0 social bookmarking applications, is an asynchronously joint approach to online information processing that engages users in collectively seeking, reviewing, gleaning and sharing valuable online resources and content for fulfilling their needs (Lin and Tsai 2011). In contrast to most of the previous research foci on individual or collaborative information searching activities (e.g., de Vries et al. 2008; Kuiper et al. 2009), this CIS activity values both individual and collaborative perspectives on the merits of information searching for quality online resources in a collective manner. In this regard, the essential goal of applying CIS activities to the educational field is to assist students in learning through collective practices while exploring the Internet.

The employment of CIS activities outlines a social-contextual scenario in which online information searching activity is conducted through a series of asynchronously communicative and negotiable person–information and person–person interactions. The application of social bookmarking to support CIS activities may constitute an ideal environment that provides opportunities for students to participate in activities of intellectual exploration, idea sharing and socially interactive collaboration (e.g., Stahl et al. 2006). Since the social context provides more opportunities for making connections to what is being learned, the properties of the interaction and meaning making have become salient aspects of the process of learning (e.g., Greeno 2006). In this regard, to understand how students react to various supportive features and peers’ feedback in CIS activities may shed light on students’ progress in learning through information searching and processing activities in which they are engaged. Furthermore, based on the perspective of activity theory that one’s thinking and activity are interactive and interdependent parts of learning (Jonassen 2002), students’ participation in CIS activities may be related to their learning through engaging in such new collective learning environments.

Participatory learning as practice in CIS activities

Participation has been viewed as one of the important prerequisites to learning in Internet-based learning environments (Hrastinski 2008, 2009). In the interactive and collaborative contexts (e.g., online discussion forums) of online learning, students are usually expected to participate in and contribute to various activities such as by expressing opinions, sharing digital resources and posting ratings for further development of peer interaction. Some empirical findings have revealed that students exhibit different levels of online participation (i.e., high, medium, low, fail) operationalized by quantitative indicators (e.g., access frequency, or the number of messages), and achieve different learning performances in terms of academic grade (Davies and Graff 2005). Furthermore, through reviewing the literature which examines the patterns and the quality of technology-enhanced interaction, Lou et al. (2001) indicated that interaction and group work may have more significant influences than individual efforts on student learning outcomes. Consequently, the effectiveness of online learning may rely on the extent to which students participate in some specific activities or events (Jin et al. 2009).

The emerging Web 2.0 applications are characterized by a number of salient features of facilitating social interaction and collaboration around the shared content, which supports the new kinds of participation for learning and literacy in Web 2.0 spaces (Merchant 2009). This phenomenon could be described as active or creative participation in the content-related discourse and mutual information exchange, which is a key theme in many conventional accounts of social interaction for learning (Lave and Wenger 1991). In this regard, using a social bookmarking system may provide an alternative platform for individuals to collect information from the open-ended Internet resources, and inspect the information recommended from a collective group of people. In the social bookmarking system, participants’ active engagement in collecting, sharing and reviewing activities underlies the success of unearthing quality online resources through the collective work of exploring the Internet. If more students are willing to participate in the process of CIS activities, this may raise the possibility of connecting those students who are willing and able to help, and also raises the possibility of obtaining relevant and useful online information resources. It could be expected that such inter-subjective interpretation relies on personal contribution and peer interaction, and may play an important role in student learning performance (Chou and Min 2009). In this regard, the application of social bookmarking to educational contexts may promote students’ learning through the practice of social participation in the distributed and collective activities of thinking and meaning-making. Consequently, in addition to the examination into the effects of implementing social bookmarking, a more systematic analysis is important to clarify participation patterns emerging in the Web 2.0-supported environment where students are engaged in learning.

Behavioral engagement in participation in CIS activities

Behavioral engagement refers to the behaviors related to one’s efforts and contributions in the involved learning activities (Fredricks et al. 2004; Ryan and Patrick 2001). An increasing number of studies have been devoted to the investigation of students’ behavior and strategy use by analyzing their activities and artifacts in the Web 2.0 context. Although many positive influences of using Web 2.0 have been reported in the literature, it is important to identify particular skills students exercise within the support of these innovative Web 2.0 applications of blogs, wikis, social bookmarking, etc. For example, by analyzing interview and blog content, Kerawalla and colleagues (Kerawalla et al. 2008) identified and characterized different kinds of blogging behaviors (e.g., blogging avoidance, resource network building, support network building, etc.). They further suggested that the effectiveness of implementing blogs may rely on how students develop and adapt their own ways of using blogs for learning. In other words, how students express their reasoning process and reflections on experiences of blogs may determine the effectiveness of blogging activities for learning (Xie et al. 2010), thus providing researchers and educators with important indicators of their cognitive strategies and understanding.

Furthermore, when a blog is used for collaborative work, students may exhibit various behavior patterns of interacting with content materials and peers. For example, by analyzing the acts of blogging recorded by log data, Hou et al. (2009) found that a group of teachers exhibited different blogging behaviors when interacting with other teachers. The behavior patterns constituted by various blog behavior indicators (e.g., the number of blogs or articles, etc.) could represent the ways users support their learning activities via blogs, and hence provide more insights into the design and implementation of fine-grained blog-based activities for learning and instruction. The study of Xie et al. (2008) further indicated the effects of blogging behaviors on students’ reflective thinking skills and learning approaches. Their findings revealed that students who had opportunities to interact with peers in blogging activities expressed a significantly higher level of reflective thinking about the activities. Consequently, thorough exploration of interactions may shed light on students’ learning experiences of adopting Web 2.0 applications, and subsequent learning outcomes of engaging in blogging activities.

By exploring the behaviors of a group of 127 junior high school students using social bookmarking, Lin and Tsai (2011) found that the students exhibited various behavioral patterns (i.e., lurker, active, quoter and critic) clustered according to a number of quantitative indicators (e.g., the number of collected and cited bookmarks, annotating personal bookmarks, and commenting on the bookmarks shared by peers) when looking for suitable online resources to perform the given task. Furthermore, the findings implied that when the students were more engaged in active participation in the CIS activities they collected more quality online information through collaborative or cooperative work, which may have led to better searching performance. Various patterns that students exhibited in their CIS activities could represent different kinds of behavioral engagement while interacting with peers to explore the Internet collectively.

Cognitive engagement in participation in CIS activities

Through the lens of constructivism, learning is an active process of how an individual integrates encountered information with pre-existing knowledge (e.g., von Glasersfeld 1989, 1993) and develops one’s knowledge through social interaction in different contexts (e.g., Cobern 1993; Solomon 1987). In this regard, learning is not only a reproduction of knowledge and skills but also a meaning-making process that the learner actively engages in. Students should construct their ways of knowing when they struggle with the conflict between discrepant events and existing personal theories. The constructivist perspectives also suggest that learning relies on meaningful interactions of learners with the content, peers and context through the process of social communication and negotiation for knowledge construction. Interactions of learners with content, peers and context have been viewed as one of the most important components of learning experiences, and learning ideally occurs in an environment where students are engaged in interactive activities of exchanging their opinions, discussing issues and collaborating to solve problems with peers.

Cognitive engagement refers to the amount of effort and type of strategies that students use in the learning tasks encountered, which is related to the effectiveness of learning (Zhu et al. 2009). Based on the framework proposed by Greene and Miller (1996), cognitive engagement could be distinguished into different levels of processing approaches to learning. One is meaningful cognitive engagement, including relatively elaborative strategies that attempt to integrate new information into the existing knowledge base for the improvement of mental representation. Another is shallow cognitive engagement, involving rote processing skills such as browsing and reading without personal judgment or reflection. Research findings have suggested that students’ persistence in exercising cognitive activities, especially those requiring high-order thinking capabilities, is likely to produce meaningful learning and facilitate content understanding, and, thus, better learning performance (Greene et al. 2004; Zhu et al. 2009).

In the research field of computer-mediated communication (CMC), asynchronous online communications and text-based discussion threads constitute an interactive context whereby participants have more time and freedom to consider an idea, reflect on their thoughts and formulate their responses (Jonassen et al. 2003; Pena-Shaff and Nicholls 2004). The extent to which participants learn mostly depends on their efforts to participate in and contribute to activities that entail learning in the context of interactive learning environments (Zhu 2006). Some of the previous CMC studies revealed that active participation and interaction, by way of analyzing the quantity of posts, messages or acts, is related to learning performance (e.g., Picciano 2002; Rovai and Barnum 2003). However, in addition to quantitative analysis, meaningful interaction for learning should be attributed to the quality of participation and interaction by examining the nature of the message content exchanged and transmitted among peers. That is, in a socially interactive context, cognitive engagement should be taken into careful consideration along with the frequency and the level of processing strategies while participating in the learning activity and interacting with peers.

In light of socio-cultural perspectives, participants’ social interactions and individual contributions to these interactions are made conscious, and are recorded in the written transcript as sequences of utterances or messages from multiple participants. These socially interactive artifacts have been recognized as data resources for research on the process of learning and the knowledge construction that is taking place (De Wever et al. 2006; Meyer 2004; Zhu 2006). Previous studies have suggested that analyzing the transcripts of message content offers a richer understanding of cognitive and social aspects of learning in particular contexts (de Wever et al. 2006; Stahl et al. 2006). Following this analytical approach, Zhu (2006) analyzed discussion messages using the method of content analysis to examine and determine students’ cognitive engagement in the context of online discussion. It can be predicted that content-related discourse, participation and socially mutual information exchange processes lead to greater conceptual understanding and learning performance (Vygotsky 1978; Lave and Wenger 1991).

Based on the aforementioned research issues, a conceptual framework was proposed to illustrate the interplay between behavioral and cognitive engagements in performing collective information searching activity, as shown in Fig. 1. Within the support of social bookmarking application, participants are encouraged to learn with exploring the Internet through a collective manner. Through participation in the collective information exploration, they need to carry out various behavioral engagements in searching, annotating, citing and commenting activities for gaging quality online resources. In the meanwhile, their exercise of cognitive engagement in the intentional and purposeful processing of encountering content may become salient in the iterative process of collective information exploration. In light of both individual and social approaches to information searching, the social bookmarking application can constitute a context of participatory learning that addresses students’ investment in and responsibility for learning with the Internet. Consequently, their participation in the CIS activity in terms of both behavioral and cognitive engagements will be critical to finding quality online resources, and achieving a better performance of the learning task.

Fig. 1
figure 1

The conceptual model of participatory learning with an collective information searching (CIS) activity

Full size image

Research purposes

Within the support of social bookmarking for collective information searching activity, this study aimed to investigate the integration of such innovative application in formal engineering course for fulfilling academic purposes. To this end, the following questions were investigated:

  1. 1.

    What kinds of engagements in terms of both behavioral and cognitive aspects the participants carried out to complete the given assignment in a collective manner?

  2. 2.

    What are the relationships between behavioral and cognitive engagements?

  3. 3.

    How are the participants’ cognitive engagement related to their collected online resources quality and examination performance?

By answering the questions above, this study addressed that both behavioral and cognitive engagements were critical to broadly represent participants’ efforts to learn with participating in the CIS activity. Furthermore, the interplay between behavioral and cognitive engagements may shed light on the benefit of behavioral interactions with social bookmarking to the advancement of cognitive efforts and strategies for participatory learning.

Methodology

Participants

This study initially enlisted 117 college students from three classes at the same school in central Taiwan. Sixteen participants were excluded from the initial pool since they missed some of the courses conducted in the research procedure. Consequently, a final sample of 101 students (89 % male and 11 % female) majoring in electronic engineering participated in this study. They were enrolled in a course introducing the principles and methods of C language programming, and were instructed by the same male teacher who had an electrical engineering major and more than 8 years teaching experience. All of the participants had the capability of performing research procedures using search engines to search the Internet, and of utilizing some prevalent applications such as web browsers, e-mail, chat messenger, and application software such as MS Office, etc. The average Internet usage among the participants was 24.05 h per week, and about 94 % of the participants did not have any relevant experience of using social bookmarking on the Internet prior to taking part in this study. Before beginning the research procedure, all of the participants were informed that all activities in the study were to be conducted via the social bookmarking system, namely WeShare, which is an online platform developed to support text-based and asynchronous interactions with peers for exploring the Internet (Lin and Tsai 2011).

WeShare in support of the CIS activity

A social bookmarking system, namely ‘WeShare,’ developed by Lin and Tsai (2011) was employed to support collective information searching (CIS) activities by way of which the participants could asynchronously manage, share and review their bookmark files of favorite sites, and create networks during the process of online information searching.

The infrastructure of WeShare is designed to meet both personal and social needs by way of some feature tools that allow users to manage and explore the online sources in different ways, as shown in Fig. 2. Employing WeShare to interpret and personalize retrieved information, the participants can add the URL of a Webpage as a bookmark to WeShare when they find some relevant Web sites involving useful information. Each bookmark has a title taken from the HTML “title” of the bookmarked page by default, and the description of the bookmark is adopted from the URL’s “description” tag if there is one, but can be edited by its initial collector. Furthermore, users can attach excerpts from the Web page, comments and tags to the bookmark, as shown in Fig. 3. The collected bookmarks are considered as one’s own property, and would not be made public. The personally-collected bookmarks are placed in the “My bookmarks” space on WeShare, which can be accessed only by the author and his/her affiliates. While bookmark collections are personally created and maintained, they are typically invisible to others.

Fig. 2
figure 2

The infrastructure of WeShare

Full size image
Fig. 3
figure 3

The interface for adding a bookmark to WeShare

Full size image

Once the original collector decides to share some private bookmarks, the shared bookmarks can be accessed and reviewed openly by other WeShare participants. Figure 4 shows the public area of WeShare which displays all of the collected bookmarks shared by WeShare participants. In the public arena of WeShare, the participants can search and browse the collected bookmarks for different given questions, and review the metadata of excerpts, comments and tags made by the original collectors. A number of user interface elements are clickable and allow participants to browse through the entire bookmark collection to see other information sources of interest.

Fig. 4
figure 4

The public area of WeShare

Full size image

When participants find some interesting or valuable bookmarks on WeShare, they can recommend and collect them in their personal collection. Since the first contributor of each bookmark is viewed as the author by default, other participants who collect the same page later are deemed as followers. Furthermore, the interaction between the author and the followers on the same given question could be facilitated by the ‘comment’ function associated with each bookmark. After reviewing the shared bookmarks, any participant can post his/her comments on the bookmarks via the comments link. Consequently, a discussion thread for a bookmark can be created by the author and followers. In sum, based on the characteristics of WeShare, the participants are able to manage their favorite sites with personal opinions, attach comments to the bookmarks contributed by peers, and join in a discussion with peers to elect more applicable sites for the given bookmark. Accordingly, the shared bookmark involves many pieces of metadata such as the time the bookmark was made, the author and any comments made on that bookmark, which give the user useful information about its context and content. In this regard, the properties of WeShare are advantageous to the development of the CIS activity of co-exploring the Internet.

Procedure

With the support of the WeShare environment, all participants carried out the CIS activity of exploring applicable online resources for the given assignment (described later) via collective work. The research procedure constituted a part of the programming course, and was conducted over a period of 8 weeks. The procedure was divided into the introductory phase (lasting 2 weeks) and the practice phase (6 weeks). In the introductory phase, an expert in information technology introduced the participants to the concepts of Web 2.0 and the purposes of applying a social bookmarking system for the activity of collective information searching, as well as demonstrating the WeShare interface employed in this study. The instructor only demonstrated WeShare, but did not either facilitate information sharing or provide feedback that reconciled the different perspectives proposed by the students in the next phase. In the introductory phase, the participants were asked to practice using WeShare to collect, annotate and share their favorite sites on the Internet, and to cite and comment on the bookmarks shared by their peers. The purpose of this introductory phase was to avoid participants’ failure in the following practice phase due to their unfamiliarity with the use of WeShare.

In the practice phase, different mechanisms of WeShare were used to support the course of C language tutorial. The course included tutorials on compilers, variables and constants, if–then switch statements, loop and function, etc. In addition to formal lectures and computerized practices, the students were encouraged to search the Internet to collect relevant examples and solutions regarding the given assignment to WeShare. They were asked to share and discuss the collected bookmarks with peers on WeShare to assess the merits of these online resources to the assignment. The students with difficulty in finding the solution were encouraged to survey the recommended bookmarks or raise their questions for seeking help on WeShare.

During the first 3 weeks of the course, the participants could search the Internet and bookmark sites in WeShare that they considered relevant to the given assignment. They could include excerpts, and make commentaries and tags on the collected bookmarks. In addition, they could decide which bookmarks they would like to share with their peers. In this stage, the features of WeShare were limited to the level of personal contributions to the CIS activity. During the last 3 weeks of the course, the participants were allowed to review, cite and comment on the bookmarks shared by their peers, as well as to select and collect bookmarks worth recommending as being most relevant to the given assignment.

A task-driven approach was employed to urge the participants to perform the activities of exploring the Internet jointly for reliable information resources. The participants enrolled in the programming course needed to complete a worksheet consisting of the given assignment. The driving assignment was to “Write a C code program to calculate and find prime numbers from 1 to 1,000, and estimate the effectiveness of the programming code in implementation.” In line with the schedule of the course progress, this assignment was reasonable and challenging for the students inexperienced in programming. The participants were informed that their performance on the assignment would be considered as part of their grade for the course. In the last week of the research procedure, all of the students needed to submit their solution to the assignment, and took a formative examination by computerized programming practice. In this regard, the course-related assignment and procedure were expected to encourage the students’ engagement in the CIS activity.

Measures

Analysis of log data for exploring behavioral engagement

All of the participants’ actions while performing the CIS activity such as bookmarking, annotating and commenting, etc. were tracked as log file data. To represent the participants’ behavioral engagement in the activity, four indicators proposed by Lin and Tsai (2011) were employed. The definition of each indicator is as follows:

  1. 1.

    Bookmarks from the Internet: The number of Web pages that the participants assembled and contributed to WeShare through searching the Internet.

  2. 2.

    Annotations on personal bookmarks: The number of annotations that the participants attached to their collected bookmarks.

  3. 3.

    Bookmarks from WeShare: The number of Web pages that the participants cited in their peers’ bookmarks by reviewing WeShare.

  4. 4.

    Comments on others’ bookmarks: The number of comments on peers’ shared bookmarks.

The frequencies of the selected indicators extracted from the log data were used to represent the participants’ investment in various contributions to the progress of the CIS activity for completing the given assignment. Furthermore, the four indicators were analyzed by the method of clustering analysis to yield different student groups, which show the patterns of students’ behavioral engagement in the activity. A two-stage clustering approach combining Ward’s minimum variance method with the K-means method was adopted. Since Ward’s method can provide the K-means method with the number of clusters as its starting point, it has been suggested that the integration of hierarchical (i.e., Ward’s method) and non-hierarchical (i.e., K-means method) methods could produce a better clustering resolution (Milligan 1985; Punj and Steward 1983).

Analysis of annotations and comments for exploring cognitive engagement

In addition to the log data of the acts of the CIS activity, the participants’ annotations and comments were adopted as data resources for qualitative analysis. This study used the term cognitive engagement to represent the participants’ cognitive efforts in and processing strategies of collecting, analyzing, interpreting and synthesizing content materials and information resources for learning from the activity.

To explore the participants’ cognitive engagement, the method of content analysis was employed to qualitatively explore metadata attached to the bookmarks (e.g., annotation and commentary), which aims to provide more in-depth understanding of the quality of the metadata. The analysis was conducted through the lens of de Wever et al. (2006), Guan et al. (2006) and Zhu (2006) who analyzed message content in terms of cognitive and metacognitive aspects within the context of online asynchronous discussion. Aligned with the purposes of this study, an analytical scheme with modified dimensions adopted from the works of Guan et al. (2006) and Zhu (2006) served as the coding system to analyze the transcripts of the annotations and comments for exploring the participants’ cognitive engagement in the process of the CIS activities. In this regard, interactions with peers are considered to facilitate the participants’ learning in a dialogical and social process in which the participants’ cognitive engagements are actively involved.

Each annotation or comment posted on the bookmarks was used as an analysis unit in this study. The author and an additional expert independently read each piece of metadata, and then assigned a category of cognitive engagement for each unit according to the analytical scheme as shown in Table 1. For example, an example response of “I think this bookmark includes more evaluative information about how to design concise programming for the question by C code. It is really helpful to the improvement of programming skills” could be categorized as “Judgmental” cognitive engagement, since it expresses a critical opinion on the bookmark. Then, all of the posts categorized by the two coders were compared to confirm the dimensions of cognitive engagement exhibited by the participants. The percentage of agreement on the coding results between the coders was used to assess the inter-rater reliability of the coding procedure. The results indicated that the inter-rater reliability of the content analysis was 84 % for the first round and 88 % for the second round. The dimensions of cognitive engagement were then analyzed by the method of cluster analysis to identify different levels of cognitive engagement (e.g., meaningful versus shallow) as suggested in the literature.

Table 1 Analytical scheme for affective and cognitive engagement in collective information searching activity
Full size table

The collected bookmarks quality for the assignment and formative examination performance

In addition to engagements in the CIS activity, the quality of the collected bookmarks for the assignment could represent the students’ searching performance of gleaning applicable online resources through collective works. To examine the quality of the bookmark collections for the given assignment, each bookmark was evaluated in terms of its relevancy, accuracy and usability which are used to critically assess and dig deep into the content involved in the Webpage (Hoffman et al. 2003). While a bookmark includes much more correct and useful materials corresponding to the given assignment, it was assigned a higher score with a range from 1 to 5 points. The method of Spearman’s pair-wise correlation analysis was employed to examine the inter-rater consistency of scoring the bookmarks. The method of Spearman’s correlation analyses has been generally employed to report internal consistency based on the scores of researcher pairs. Consequently, the results of Spearman’s correlation between two experts revealed that the coefficients of relevancy, accuracy and usability were 0.87, 0.85 and 0.80 for the assignment. In addition, a computerized practice took place at the end of the research procedure. The teacher employed a 100-point scale to score students’ performance of the computerized practice test. Since this study was embedded in the formal course, the students’ performance on the formative examination was regarded as a part of learning outcome for further analysis.

Research findings

Descriptive statistics of behavioral engagement in the CIS activity

Table 2 reveals the descriptive results of four indicators as personal contributions to the CIS activity. The results revealed that the participants collected more bookmarks from WeShare than from the Internet. In addition, they tended to frequently make comments on peers’ bookmarks. The statistical findings indicate that these college students were inclined to collect information resources shared and recommended by peers on WeShare, as well as to make comments on the shared information resources.

Table 2 Descriptive statistics of behavioral engagement in the CIS activity (n=101)
Full size table

Students’ behavioral engagement in the CIS activity

To explore the participatory patterns of behavioral engagement in the activity, this study employed the method of two-stage clustering analysis akin to the work of Lin and Tsai (2011). Ward’s method was adopted to generate possible cluster solutions first. Subsequent sets of cluster solutions were then analyzed by the K-means method for aggregating different CIS indicators into possible patterns of behavioral engagement in the activity. An analysis of variance (ANOVA) examining the inter-cluster differences across the CIS indicators (i.e., “Bookmarks from the Internet,” “Annotations on personal bookmarks,” “Bookmarks from WeShare,” and “Comments on others’ shared bookmarks”) revealed that the four-cluster solution yielded the clearest distinctions among and provided more meaningful explanations for the different patterns of behavioral engagement.

Table 3 shows the numbers of participants, mean values of the CIS indicators in each cluster, and the comparisons of the post hoc tests. The results of the ANOVA analyses revealed that there were significant differences among clusters for all of the CIS indicators of “Bookmarks from the Internet,” “Annotations on personal bookmarks,” “Bookmarks from WeShare,” and “Comments on others’ shared bookmarks.”

Table 3 The clusters of users’ participatory patterns of behavioral engagement in the CIS activity
Full size table

Furthermore, the results of a series of post hoc tests (Scheffé tests) support that the four clusters could be employed to interpret the differences in the participatory patterns of students’ contributions to the CIS activity within the context of WeShare. Based on the results of the cluster analysis, the participants could be categorized into four major groups which exhibit distinctive characteristics in the composition of the participatory patterns of engaging in the CIS activity. Referring to the previous work of Lin and Tsai (2011), these four groups are re-labeled and interpreted as follows:

Hitchhiker

As shown in Table 3, cluster 1 includes 27 participants accounting for 26.7 % of the study sample. Compared with other clusters, the frequencies of “Bookmarks from the Internet” and “Annotations on personal bookmarks” exhibited by cluster 1 were significantly lower than those of any other cluster. Cluster 1 also had significantly lower frequencies of “Bookmarks from WeShare” and “Comments on others’ shared bookmarks” than cluster 3 and 4. However, when compared with cluster 2, cluster 1 had a higher frequency of the indicator “Comments on others’ shared bookmarks.” These results reveal that the participants in this group tended to exert minimal effort to collect information resources by searching the Internet, but tended to comment on or cite peers’ bookmarks while engaging in the CIS activity using WeShare. It could be suggested that these students may have tended to ‘hitch a ride’ during the activity, and so can be viewed as “Hitchhikers.” This group of students could be akin to a combination of “Lurker” and “Quoter” in the work of Lin and Tsai (2011).

Individualist

The second cluster includes 38 students accounting for 37.6 % of the study sample, which is the largest group among the four clusters. They exhibited significantly higher frequencies than cluster 1 for the dimensions “Bookmarks from WeShare” and “Annotations on personal bookmarks,” which to some extent reveals a reverse pattern to that of cluster 1. Furthermore, the students in cluster 2 had the lowest frequency of the dimensions “Bookmarks from WeShare” and “Comments on others’ shared bookmarks”. In this regard, these participants tended to invest more efforts in searching, collecting, and annotating bookmarks from the Internet themselves rather than in consulting the publicly shared resources on WeShare. They revealed an individualistic approach to the contribution of information sources, and so could be labeled as “Individualist” with respect to their behavior throughout the CIS activity.

Active

The third cluster accounts for 22.8 % of the study sample (n = 23) and has the highest frequencies for the dimensions “Bookmarks from the Internet,” “Annotations on personal bookmarks” and “Bookmarks from WeShare.” The students in this cluster reflect a significantly higher level of effort than cluster 1 on all CIS behavioral dimensions. They also had significantly higher frequencies than cluster 2 on the dimensions of “Bookmarks from WeShare” and “Comments on others’ shared bookmarks.” The students in this cluster could be deemed as an “Active” group who energetically participated in the different CIS activities.

Commentator

Finally, the participants in cluster 4 (n = 13) constitute the smallest group. Akin to cluster 3, the students in this cluster had significantly higher frequencies than those in cluster 1 for all CIS dimensions, and than those in cluster 2 for the dimensions “Bookmarks from WeShare” and “Comments on others’ shared bookmarks.” More particularly, the students in this cluster had the highest frequency for “Comments on others’ collections” when compared to other clusters. Regarding this aspect, the participants of cluster 4 could be viewed as the “Commentator” group who tended to comment on peers’ shared bookmarks.

Descriptive statistics of students’ cognitive engagement in the CIS activity

Based on the analytical scheme adopted in this study (see Table 1), the students’ annotations and comments on the shared bookmarks were analyzed by the method of content analysis, and then categorized into different levels of cognitive engagement. As shown in Table 4, the results reveal that the students showed varied strategies of cognitive engagement while interacting with social bookmarking to perform the CIS activity. Furthermore, the results of Table 4 indicate that affective expressions were mostly exhibited when annotating and commenting on the bookmarks. However, the participants had relatively slight exercise to perform tutorial strategy in the CIS activity.

Table 4 Descriptive statistics of cognitive engagement in the CIS activity
Full size table

Cluster analysis of students’ levels of cognitive engagement in the CIS activity

According to the proposition of Greene and Miller (1996) that cognitive engagement could be dichotomized as meaningful and shallow, this study adopted a pre-defined two-cluster solution for the cluster analysis of cognitive engagement as “Deep” and “Surface” levels. The dimensions of cognitive engagement were purposefully aggregated into two groups by the method of cluster analysis. Then, the differences in the dimensions between the two groups were examined by a series of simple t-tests to differentiate and interpret the levels of cognitive engagement. Table 5 shows the numbers of participants, the mean values of the dimensions in each cluster, and further comparisons by way of independent t-tests. The results reveal that there were significant differences between clusters for the “Irrelevant,” “Affective,” “Literal,” “Explanatory,” “Judgmental” and “Reflective” dimensions. According to the results shown in Table 5, the “Deep” cluster included 40 students accounting for 39.6 % of the sample, which had lower frequencies of “Irrelevant,” “Affective” and “Literal,” as well as higher frequencies of “Explanatory,” “Judgmental” and “Reflective” cognitive engagement than the students in the “Surface” cluster (61 students, 61.4 %).

Table 5 The levels of students’ cognitive engagement in the CIS activity by use of K-means cluster analysis
Full size table

The results of the cluster analysis reveal that the students exhibited distinctive characteristics in the composition of the cognitive engagement patterns. Those students in the “Deep” cognitive engagement group demonstrated a relatively higher level of cognitive effort for the strategies of explanation, judgment and reflection. In contrast, the students in the “Surface” cognitive engagement group expressed a relatively lower level of tactics usage such as irrelevant, emotional and literal responses.

The associations among students’ behavioral and cognitive engagements

In Table 6, the cross-tabulation of categories of behavioral engagement at the levels of cognitive engagement is presented. The 4 × 2 table shows behavioral engagement (“Hitchhiker,” “Individualist,” “Active” and “Commentator”) in rows and cognitive engagement (“Deep” and “Surface”) in columns. A Pearson’s chi-square test was performed to identify the association between the students’ participatory patterns and cognitive engagement.

Table 6 The association between students’ behavioral and cognitive engagements in the CIS activity
Full size table

The results in Table 6 reveal a significant association between participatory patterns and cognitive engagement during the CIS activity; namely, the students in the Hitchhiker and Individualist groups tended to invest surface cognitive engagement (n = 23, 27, respectively) while those in the Active and Commentator groups were more likely to adopt deep cognitive engagement (n = 16, 9, respectively). It could be suggested that students’ participatory patterns and cognitive engagements are highly associated.

Comparisons of the students’ collected bookmark quality and formative examination performance between the levels of cognitive engagement

Table 7 reveals the comparisons of students’ bookmark quality for the assignment and formative examination scores between different levels of cognitive engagement in the CIS activity. The quality of bookmark collected for the assignment was assessed by two additional experts. Higher scores may signify students’ capability to glean quality online resources through the CIS activity. The results reveal that students with deep cognitive engagement had significantly higher scores than the others on the evaluative standards of accuracy and usability for the assignment. In addition, the students with deep cognitive engagement in the CIS activity significantly outperformed the others in a formative test of computerized practice. According to the findings above, in the CIS activity students who exercised more advanced strategies tended to perceive the merits of the bookmarks suitable for the assignment, and had better assignment performances.

Table 7 The collected bookmark quality and formative examination scores between deep and surface levels of cognitive engagement
Full size table

Discussion and conclusion

The application of social bookmarking to support collective information searching (CIS) activities emphasizes the aspects of individual and collaborative online information problem solving through its active and interactive nature (Lin and Tsai 2011). This innovative Web 2.0 application can offer students a technology-supported collective inquiry context which sustains specific features of learning environments helpful to the improvement of cognitive engagement (Blumenfeld et al. 2006). On the one hand, students’ cognitive engagement may come with their active, constructive and collective work of searching for solutions and joining in asynchronous dialogue to solve information-related problems in such a new interactive context. On the other hand, the ways students exercise different mechanisms of social bookmarking and their investment in its use may represent their situational interest and behavioral engagement in the activity, which in turn may boost the employment of higher-level cognitive strategies and self-regulation. As previous findings have indicated that active participation in CIS activities is critical to the elicitation of peer feedback and the quality of online resources (i.e., Lin and Tsai 2011), this study aimed to explore the relationship between participants’ behavioral and cognitive engagements in a CIS activity for completing searching tasks.

Given the increasing exposure to online resources and Web 2.0 applications, accessing online information need not be an individual effort, but inherently involves collaborative and collective activities (Hansen and Jarvelin 2005). The results of analyzing activity log data firstly identified four participatory patterns of “Hitchhiker,” “Individualist,” “Active” and “Commentator” among a group of college students in this study. These participatory patterns revealed that the students exhibited different genres of behavioral engagement in the activity. In contrast to the participatory patterns identified in the work of Lin and Tsai (2011), two new patterns of “Hitchhiker” and “Individualist” are proposed since the students in this study exhibited somewhat different endeavors while carrying out the activity. For example, the students categorized in these new patterns invested particular efforts in performing different sets of CIS activities. Differing from the “Lurker” category identified from exploring the junior high school sample in the previous study, these college students seemed to exhibit a more active approach to different aspects of the CIS activity rather than merely lurking. However, the findings of this study revealed that only about one-fifth of the students (i.e., the 23 participants in the “Active” group) could take full advantage of WeShare to collectively seek and survey online resources. It is suggested that the employment of Web 2.0 applications may not necessarily ensure a special attraction for students’ behavioral engagement in regular use of these innovative applications for learning. In addition, the students’ unfamiliarity with the usage of WeShare and limited experience of executing CIS activities may also have inhibited their willingness to make further contributions to the activity. In this regard, it is necessary to provide students with more opportunities to become familiar with the use of such innovative tools for academic purposes.

In addition to the recognition of the college students’ behavioral engagement in the CIS activity, this study further identified their cognitive engagement by their personal annotations and comments during the activity. By analyzing the transcripts of the annotations and comments, the results revealed that the students expressed diverse cognitive engagement in the activity. “Affective,” “Judgmental” and “Irrelevant” strategies were frequently adopted. Further cluster analysis of cognitive engagement could be classified into dichotomous levels of “Surface” and “Deep” engagement. Those students with a relatively deep level of cognitive engagement tended to frequently adopt “Explanatory,” “Judgmental” and “Reflective” strategies, whereas those with relatively surface level engagement usually employed “Irrelevant,” “Affective” and “Literal” strategies while performing the CIS activity. The deep level of cognitive engagement found in this study implies that the students could provide explanations, voice their opinions, evaluate peers’ shared information and reflect on their understandings. In contrast, the surface level of cognitive engagement denotes that the students tended to offer irrelevant or factual information, and express their feelings unrelated to the subject. Through investigation of students’ cognitive engagement, researchers and educators can understand students' efforts and strategies for dealing with online information resources collectively. In addition, further cluster analysis offers a better understanding of which strategies could be attributed to a relatively higher level of cognitive engagement. As meaningful cognitive engagement is suggested to produce better learning outcomes (Greene et al. 2004), to stimulate the occurrence of such engagement, students may benefit greatly from participating in CIS activities.

This study provides some evidence of the role of students’ mutual and reciprocal actions in their extended engagement in cognitive activities in a CIS activity. The result of chi-square analysis revealed the relationships between behavioral and cognitive engagements in this CIS activity in the context of social bookmarking. The students with behavioral engagements of “Active” and “Commentator” displayed a relatively deep level of cognitive engagement, whereas those students with behavioral engagements of “Hitchhiker” and “Individualist” showed a relatively surface level of cognitive engagement. This finding runs parallel to previous studies suggesting that students’ cognitive engagement is more animated and advanced when interacting with peers (Greene et al. 2004). As social bookmarking provides students with a technology-supported environment to explore the Internet in a collective manner, their behavioral and cognitive engagements would intertwine through mutual and reciprocal interactions during the activity. Furthermore, as collaborative interaction is helpful to critical evaluation of online information resources (Butler and Lumpe 2008), it is suggested here that active participation in CIS activities may stimulate a relatively higher level of cognitive engagement while evaluating online information collectively. Consequently, it could be suggested that different learning outcomes may be achieved according to various patterns of students’ active behavioral and cognitive engagement with the collective work of exploring the Internet.

The findings of this study also indicated the levels of cognitive engagement were related to the collected bookmark quality and formative examination performance. The students having more advanced cognitive strategies in the CIS activity tended to become aware of valuable online resources for the assignment. They also got higher scores for the computerized programming practice than others with surface cognitive engagement. Since this study did not employ an experimental design, these results could not be attributed to the effect of integrating CIS activity into the course for academic learning. However, the concept of participatory learning has become salient in the CIS activity for supporting learning with exploring the Internet. The application of social bookmarking application could provide an alternative way to support academic learning and instruction in a socially interactive context, and help learners understand and develop relevant strategies to deal with the quantity and quality of online information.

In this study, 216 bookmarks were individually stored by the participants in WeShare, and there were 52 distinct sites located on different URLs among these collected bookmarks for the given assignment. Among these distinct bookmarks, 14 were cited more than 15 times and followed by 5 distinct participants at least. Furthermore, these bookmarks with relatively high citations revealed good quality in terms of their relevancy, accuracy and usability. Although this study addressed the concept of participatory learning by investigating the students’ behavioral and cognitive engagements in the CIS activity; however, investigating the properties of the collected bookmarks may inform an alternative approach to research on CIS activity.

Participation has been viewed as a critical part of online learning owing to its positive effects on various learning outcomes (Davies and Graff 2005; Hrastinski 2009; Michinov et al. 2011). However, examining online participation remains a key issue since most studies tend to rely simply on frequency counts as measures of participation (Chan and Chan 2011). Such participation measurements may fail to explicate the considerable benefits of technology-enhanced environments for learning within socially interactive contexts (Hrastinski 2008, 2009). Aligned with the perspectives of online participation, more complex and multiple dimensions are necessary for a better understanding of online learning. The employment of CIS activities supports the idea of “folksonomy,” allowing students to participate in the process of annotating and categorizing content, which may amplify the potential of seeing how others interpret and value information resources that we share (Morrison 2008). According to the findings mentioned above, it is suggested that exploring students’ active engagement from both behavioral and cognitive aspects is helpful in clarifying the perspectives of participation in learning through behavioral and psychological strategies and investments in CIS activities. Through iterative processes of participatory behaviors and cognitive engagement, students are expected to become more critical of and thoughtful about open resources while searching the Internet. Accordingly, they may acquire better learning materials from the Internet, undertake meaningful cognitive engagement with and interaction between themselves and the content, as well as perceive the merits of Internet-based environments in support of the learning process.

Limitations and future research

The application of social bookmarking assists students in keeping online information they might want, as well as sharing and connecting with like-minded peers. Since more and more content is being presented online, educators need to assist students in developing the skills to collect, store, and retrieve relevant information effectively. Furthermore, educators have to equip them with the ability to work closely with others for the collaborative construction of knowledge. Adopting social bookmarking redefines the ways in which we think about learning and teaching by way of online information searching for inquiry- and problem-based activities. The findings of this study imply that, within the scaffold of social bookmarking for exploring the Internet, educators need to encourage students to become more active contributors rather than passive users of online information for learning. Further research has to embed instructional methods in CIS activities to promote participation rates and help students to develop new literacies in the Web 2.0 age.

Referring to the findings of this study, one may argue that the average frequency of actual bookmarks from the Internet and WeShare may seem rather low during the 6 weeks of research procedure. This may result from the approach to information searching and the characteristics of the assignment. Although all the students were asked to perform the assignment on WeShare, not all students engaged in explicit searching. At the other extreme, some students could be capable of finding all the information they needed by searching the Internet on their own. However, within the support of the social bookmarking application, the students could not only aggregate more refined resources to perform the assignment, but also participate in iterative interactions with peers for meaning negotiation and knowledge construction. Many significant differences of behavioral and cognitive activities existed among the students, revealing the role of individual differences in information searching in participatory learning within the CIS activity.

Furthermore, although the assignment conducted in this study allowed for different solutions, it should be considered a relatively fact-oriented question with definitive answers. Since all of the students were attending a programming course for their first time, the assignment may be considered challenging enough for these novice programmers. However, the characteristics of the assignment may have limited the students’ willingness and efforts to perform the collective information searching activity. Future studies need to carefully investigate the potential of social bookmarking application for assisting students in performing an open-ended and project-based task.

In addition, although there were many significant relationships between the behavioral and cognitive engagements in the process of the CIS activity, the case of discussion threads attached to bookmarks for further discussion and negotiation was not common. Since it requires a lot of work to compile the collected bookmarks for a CIS activity, students’ perceived information overload may have hindered their participation and cognitive engagement. It is therefore suggested that the students needed more time to perform the activity, and other facilitators such as instructors or formative feedback on how they could enhance vital interactions and reduce perceived information load need thoughtful consideration in future research.

Unlike the use of wikis or blogs for content creation, the application of social bookmarking focuses primarily on creating connections between content and people. An exploration of the networked content–content, content–user and user–user relations may deepen our understanding regarding how these iterative interactions influence students’ choice of information resources and group formation through participating in CIS activities. Accordingly, there are many critical research issues raised with such innovative learning context. How does a student’s learning trajectory alter in accordance with comment and identity received from participating in the CIS activity? How does an interest group develop through iterative interactions in the CIS activity? To explore such research issues needs more fine-grained and specific ways of conducting both quantitative and qualitative analyses to depict students’ learning in more detail. Some specific methods such as sequential analysis and social networking analysis are applicable for analyzing dynamic process of the CIS activity. These analytical techniques and approaches could be employed in future research to explore which online resources are valuable, and the composition of group affiliation while students participate in CIS activities for learning specific topics related to personal interests. Consequently, these identified information resources and members could be critical to the facilitation of students’ learning and the development of learning communities.

Employing social bookmarking to engage students in CIS activities really challenges educators and researchers to rethink the way in which students treat the information they find, to redefine the process of personal cognitive operation of socially negotiated content, and to examine its potential for the attainment of more and better information in a communal model. Inevitably, students have begun to develop a different relationship with the Internet that has raised numerous implications for teaching and learning. These Web 2.0 applications may not be necessary for effective learning, but demand that educators and researchers should recognize their potential for supporting the reformation of content and curriculum for improving students’ learning. An increasing number of studies are devoted to research on the educational potential of these innovative technologies, but schools have been slower to consider the use of Web 2.0 applications for teaching and learning in the classroom (Richardson 2006). This study suggests that instructional design can combine different salient mechanisms of various Web 2.0 applications in line with the objectives of teaching and learning.

Based on the aforementioned descriptions, it is proposed that social bookmarking can not only be used as a research tool for investigating collective information behaviors, but also as an instructional tool for engaging students in participatory learning. In addition to the concern about one’s own learning progress, students need to be aware of their responsibility to contribute to the participation in CIS activities. The more shared information resources attached with one’s opinions for academic purposes, the more easily learners can find and connect to the learning resources they need and desire. These metadata that others apply to the content of different subjects may provide students with various experiences and perspectives on learning about what they are really interested in. Consequently, these reusable information resources could constitute a database which includes more fine-grained free online learning resources, and the more students who contribute their efforts to CIS activities, the more valuable learning resources and experiences will be generated.