1 Introduction

Community colleges are one of the vocational and technical education institutions which help learners to gain competencies for a specific occupation. Several reports have been prepared to empower vocational and technical education. The reports not only focus on the general problems and solutions about vocational and technical education but also specific problems in community colleges. Additionally, similar problems expressed in many countries around the world as the United States of America (American Association of Community Colleges 2012; Dougherty et al. 2017), Korea (Grubb et al. 2009), China (Gallagher et al. 2009), and Turkey (Ministry of Development 2014).

As for the community colleges, the most important problems expressed in the reports and literature are students’ lack of a proper foundation (Dougherty et al. 2017), students’ lack of transferring theoretical knowledge and skills to the different contexts in the real life (American Association of Community Colleges 2012; Cisneros 1996; Grubb et al. 2009; Tierney 2014), low levels of academic achievement and motivation among students (Dougherty et al. 2017; Grubb et al. 2009), use of traditional methods for instruction (Şahin and Fındık 2008). These problems may reduce the quality of the teaching/learning process of community colleges and this may lead to students not gaining sufficient competencies about a specific occupation (Grubb et al. 2009). As one of the solutions to solve these problems, learning environments and instructional processes can be improved by applying innovative teaching methods and technologies.

To be successful in school courses, students should be cognitively and socially engaged in course activities (Tinto 2011). One method to achieve that is the collaborative problem solving method. It is based on the constructivist learning principles, which help educators to engage students cognitively in the coursework and equip students with the competencies needed in the business sector. Problem solving activities can help students to develop their learning skills and enhance interaction between students and the environment (Gross 2017). Previous studies concentrated on learning environments where collaborative problem solving method and problem based learning principles were used revealed the positive effect of these learning environments on metacognitive skills of higher education students (Shutimarrungson et al. 2014; Şendağ and Odabaşı 2009), course engagement of higher education students (Delialioğlu 2012; Fukuzawa and Boyd 2016), and course achievement of higher education students (Hou et al. 2016; Korucu and Cakir 2018; Roberts 2017) and K-12 education students (Horak and Galluzzo 2017). Furthermore, to improve students’ academic achievement and engagement, using web 2.0 technologies can be beneficial. Web 2.0 technologies are web based applications that help users to consume, to create, and to share the content, to work together, and to communicate and interact with other users easily (Franklin and Van Harmelen 2007). In the literature, using web 2.0 technologies can be beneficial for students to improve their performance in school courses at a different level of education such as k-12 education (Huang et al. 2016; Pietikäinen et al. 2017), and higher education (Tsai and Tang 2017; Virtanen and Rasi 2017).

Limited studies are focusing on using technology-supported collaborative problem solving method to improve students’ achievement and engagement in the context of technical and vocational education (e.g. Jabarullah and Hussain 2019; Lyons 2008; Ozkara and Cakir 2020; Papantoniou and Hadzilacos 2017; Roberts 2017). As community colleges have an important role in providing skills and knowledge for students to enter the job market after formal education, it can be useful to design a learning environment with collaborative problem solving method and incorporating web 2.0 technologies to support this learning environment. Thus, the outcomes of this study can help instructors at community colleges to design courses which employ collaborative problem solving method supported with web 2.0 technology tools. In addition, this study can be beneficial to enhance students’ knowledge and skills as this learning environment provide more interaction opportunities between students, content, and instructor. Finally, this study can contribute to the body of knowledge on designing constructivist learning environments and using web 2.0 technologies in these learning environments. Accordingly, the main question of this study is “what is the effect of the learning environment designed based on collaborative problem solving method and supported by web 2.0 technologies on students’ academic achievement and engagement?”.

2 Literature review

2.1 Collaborative problem solving

Collaborative problem solving is a method where students work collaboratively, engage in ill-structured problem solving, and finally, reach a solution for the problem and reflect on their entire learning process. The collaborative problem solving method offers a comprehensive guideline incorporating the principles of problem based learning and collaborative learning (Nelson 1999). Collaborative problem solving is based on constructivism. Therefore, learning occurs around solving an ill-structured problem. Different activities such as presentation of the problem, defining the problem, presenting similar solutions, setting the hypotheses, using different information resources and cognitive tools, working with classmates, and receiving support from the environment are applied in the problem solving process (Jonassen 1999). Like Jonassen’s constructivist learning environment model, there are similar steps in collaborative problem solving. These steps are to build readiness, form and norm groups, determine a preliminary problem definition, define and assign roles, engage in an iterative collaborative problem solving process, finalize the solution or the project, synthesize and reflect, assess products and processes, and provide closure (Nelson 1999).

There were several research conducted on problem based collaborative learning. According to the results of prior research (Podges et al. 2014; Günter and Alpat 2017; Balendran and John 2017; Lin 2017; Korucu and Cakir 2018), problem based collaborative learning is an effective way to increase students’ achievement. Furthermore, students were more engaged in the problem based learning environment (Chou and Chen 2008; Malhiwsky 2010; Marra et al. 2014; Alioon and Delialioğlu 2017). Another important contribution of problem based collaborative learning is about developing higher order thinking skills of students such as problem solving, analytical thinking, and other cognitive abilities. For example, Argaw et al. (2017), Gholami et al. (2016), Ismoyo (2017), Wijnen, et al. (2017), McCrum (2017) examined the effect of problem based learning on different cognitive abilities of the participants and found that problem based learning could improve students’ higher order thinking skills. Through the findings of those studies, Nelson’s (1999) collaborative problem solving steps as indicated above were adopted for the current study.

2.2 Web 2.0 technologies

In the constructivist approach, technology can be used by students for problem solving, and students can make their own meaning with the help of technology (Jonassen and Reeves 1996). Therefore, technology is not entirely used for planning and presenting the course content as in the conventional methods, rather it is used for the product oriented to construct new knowledge in constructivist learning environments. In this regard, web 2.0 technologies can be utilized with the collaborative problem solving method. Web 2.0 technologies “allow users to (1) share knowledge through collaborative editing, communicating, publishing, and commenting and/or to (2) dynamically change the content of knowledge published on the web” (Kale and Goh 2014, p. 42). Additionally, web 2.0 technologies are driven and controlled by user contributions (Ajjan and Hartshorne 2008). Thus these technologies change the way of collaboration and sharing of knowledge and lead users to construct knowledge through interactions. As these technologies allow users to participate in collaborative activities (Huang et al. 2013), the use of these technologies provides new ways to support and enhance the learning process in higher education (Ajjan and Hartshorne 2008). The affordances of these technologies overlap with the collaborative problem solving learning principles as collaboration, active participation in the knowledge construction, interaction with students, instructor and content, and so on. There have been studies conducted on the educational contributions of learning environments supported by web 2.0 technologies. Malhiwsky (2010) has integrated different web 2.0 technologies into the course and investigated students’ views at the community college. Students have expressed views about the usefulness and ease to use of web 2.0 tools and their satisfaction with web 2.0 tools. Kay and Kletskin (2012) have used problem based video podcasts in the course at the university and students evaluated video podcasts as useful, easy to use, and effective tools. Jaffar (2012) has explored medical students’ views about Youtube videos within problem based learning. Students have viewed Youtube as a social network site and learning resource and helped them to learn the course content. Ioannou et al. (2016) have designed a problem based learning environment in the postgraduate course using tablets, smartphones, and Facebook and have examined students’ experiences. The researchers have found that Facebook is useful for recordkeeping and communication in this environment. Huang et al. (2016) have explored the effect of blog and microblog in problem based learning environment on seventh-grade students’ achievement. The results have indicated that using a microblog in problem based learning is better than using a blog for increasing achievement. There have been similar research results about positive views on web 2.0 tools in the courses (Majid 2014; Uzunboylu et al. 2011). As discussed in the literature, web 2.0 technologies can be utilized in the steps of collaborative problem solving to improve students’ academic competencies and engagement.

In summary, using collaborative problem solving method and web 2.0 technologies in the learning environments have a positive impact on students’ learning performance and engagement. As the literature has limited studies about solving instructional challenges encountered in the context of community colleges, using the collaborative problem solving method and web 2.0 technologies in the learning environment at the community colleges and evaluating the impact of this learning environment can contribute to the literature.

2.3 Purpose of the study

The purpose of this study was to determine the effect of a learning environment designed based on the collaborative problem solving method and supported by web 2.0 technologies on students’ academic achievement and engagement. Three groups were selected for this study; two experimental groups and one comparison group. One of the experimental groups worked in the learning environment where the collaborative problem solving method was implemented and web 2.0 technologies were used to solve ill-structured computer programming problems, and the other experimental group used face to face communication and desktop software to solve ill-structured computer programming problems while the students in the comparison group the course was taught with the traditional methods such as lecturing and demonstration. Following are the research questions that guide the current study;

  1. 1.

    Is there a significant difference between the experimental groups and comparison group in terms of;

    1. a.

      Academic achievement?

    2. b.

      Academic engagement?

3 Method

3.1 Research design

In this study, a pretest–posttest quasi-experimental design was used. The experimental design of the study was presented in Table 1.

Table 1 Experimental design of the study
Full size table

This experiment was carried out with three groups (two experimental groups and a comparison group) in the Object-Oriented Programming I-II courses which were in the curriculum of computer programming at community college. The groups were assigned to conditions randomly. Experimental Group 1 (EG1) enrolled in Object-Oriented Programming I-II courses where the collaborative problem solving method was implemented was supported with web 2.0 technologies to work on ill-structured problems, and the Experimental Group 2 (EG2) used face to face communication and desktop software to work on ill-structured problems. The same courses in the comparison group (CG) were taught with traditional methods such as lecturing, demonstration-performance. Moreover, the students in the comparison group worked on structured problems and their solution process which was differed from collaborative problem solving method. The instructor lectured the content and explained the programming concepts and terms using notes and slides. In addition, the instructor demonstrated the coding activity related to the topic for each week in the lab session, then students individually applied the same codes in the lab session. Moreover, these problems were well defined, had one solution, and required the transfer and implementation of the programming concepts and rules for the solution (Jonassen 1997) in each week. Thus, students were taught by the traditional method and engaged in a structured problem solving process. As a consequence three groups enrolled in the same courses at the department of computer programming. The experimental groups were selected from a community college. Due to the lack of a sufficient number of students in that community college, the comparison group was selected from another community college in the same university. At the beginning of the experiment, an achievement test was administered to students in all three groups as a pretest. After the experiment finished, the achievement test and an engagement scale were administrated to the students as the posttest. However, the projects which were prepared by the experimental groups were evaluated according to the project evaluation form.

3.2 Participants

This study was carried out during the 2015–2016 academic year on a total of 94 students. The students who registered for the Object-Oriented Programming I-II courses at the department of computer programming from two public community colleges in the western part of Turkey participated in the present study. The number of participants is cross-tabulated by group and gender in Table 2.

Table 2 Distribution of the participants by the groups
Full size table

Table 2 shows that there are 36 students in the EG1, 33 students in the EG2, and 25 students in the CG. In terms of gender, the number of male students is higher than female students. Furthermore, the students were not randomly assigned to the groups. Instead, intact groups were randomly assigned to the conditions. In addition, students’ level of pre-knowledge about programming was taken into consideration to ensure the equivalence of groups. To do this, the Basics of Programming course scores of the students were taken from the student administration office. Then, ANOVA was conducted to test whether there was any significant difference in terms of pre-knowledge about programming between the experimental groups and the comparison group. The analysis result showed that there was no significant difference between the selected groups F(2, 91) = 2.081, p > 0.05. As a result, the equivalence of the groups was ensured before the experiment.

3.3 Data collection instruments

The data were collected with a personal information form, an achievement test, and an engagement scale. Personal information form consisted of questions such as gender, age, ownership of information and computer technologies (computer, tablet, smartphone), access to the internet.

Student achievement

The achievement test was used both before and after the experimental process to the students in all groups to compare groups on common ground. However, to compare the two experimental groups in terms of academic achievement, half of the academic achievement test score (%50) and half of the group project evaluation score (%50) were taken into consideration. The academic achievement test was comprised of 40 multiple choice questions to measure students’ knowledge level of object-oriented programming. Some of the question types were comprehension, memory retention, and retrieval related to programming knowledge, while other types were analysis and restructuring of computer programming. It was designed by the researchers by preparing an instructional objectives table, receiving expert reviews, and administering a pilot test. Based on the results from the pilot test, reliability, item difficulty, and item discriminations were calculated. The Kuder-Richardson-20 (KR-20) was calculated for the reliability of 0.85. The item difficulties ranged from 0.36 to 0.79 with discrimination of larger than 0.20 on all but six items. These items were evaluated by the researchers and experts. Based on the expert suggestions, the necessary revisions were done. Finally, the results of reliability and the item analysis of the achievement test showed that the achievement test could be useful for the study.

Project evaluation

A project evaluation form was prepared by the researchers to evaluate the group projects in the experimental groups. These projects were evaluated according to the criteria on the form. Then a group score was acquired and each member of the groups was given the same scores. Before preparing the form, the researchers explored course outcomes, the minimum requirements of the project, the problem solving process, and the software development process. Then the researchers prepared items and received opinions from three experts. Based on the expert suggestions, the researchers made slight modifications and prepared the final form. The form had two sections; a check-list about the adequacy of projects and a software rating scale. The purpose of the check-list was to understand whether projects meet the requirements explained in the course syllabus. The software rating scale was a five-point Likert type scale with 16 items to rate software ranging from poor (1) to excellent (5). Seven items in the scale were related to the problem solving process while nine items were related to the quality and accuracy of the software. Therefore, students in the experimental groups were evaluated in terms of programming knowledge and problem solving related to the software development process by this form similar to the achievement test. After the experiment, the researchers and instructor evaluated students’ projects separately from each other. The reliability of the project evaluation was ensured with an inter-coder reliability value. Inter-coder reliability is viewed as a relationship between coders’ judgments of project scores (Tinsley and Weiss 1975). Therefore, correlation analysis was conducted between scores which were given by two different coders. It was found that there was a significant correlation between the two raters’ scores, (r = 0.86, p < 0.001). It was concluded that this was acceptable for the reliability of the project evaluation.

Student engagement

The Community College Survey of Student Engagement (2016) was adapted into Turkish to measure students’ level of engagement. This scale was modified from the National Survey of Student Engagement (2013) with permission from Indiana University. The items on the scale were about the educational practices, time effort on educational tasks, quality of the relationship between students and college, and extracurricular activities at community colleges. The Community College Survey of Student Engagement (2016) had five subfactors as active and collaborative learning, student effort, academic challenge, student-faculty interaction, and support for learners. As a result, this scale measured how much effort students devoted to the courses, how much time students spent on activities, how many educational practices students did and students’ engagement level in challenging mental activities, and communication quality between students and instructors.

The researchers adapted the scale with the permission of the institution. All sections of the scale were examined by the researchers. The active and collaborative learning subfactor of the scale was adapted into Turkish within the purpose of the study since the research was conducted to find out the students’ level of involvement in the courses. The items were about activities done in and out of class, discussions with groupmates about the project, the use of technology for the course, working on the project, communication, and collaboration with groupmates and instructor. Sample items in this section are “Asked questions in class or contributed to class discussions”, “Discussed ideas from your readings or classes with instructors outside of class”, and “Worked on a paper or project that required integrating ideas or information from various sources”. The translated scale was applied to 278 students at community college for validity and reliability. Exploratory factor analysis was conducted for construct validity. According to the analysis, the active learning subfactor had eight items explaining 26.58% variance and the collaborative learning subfactor had five items explaining 10.79% variance. Item total correlation was used for the item analysis. The calculated correlations were between 0.22 and 0.65. According to the analysis, only one item was seen as problematic. This item was reviewed by the researchers and then it was decided that this item could be used with a revision in the scale. For the reliability of the scale, Cronbach’s Alpha internal reliability coefficient was found 0.80 for the active learning subfactor, 0.71 for the collaborative learning subfactor, and 0.90 for the whole scale. These values were regarded to be convenient for the study.

3.4 Development of the learning environment

In order to design the learning environment, the collaborative problem solving method and web 2.0 technologies were utilized. The students worked in groups to solve the problems related to given situations and develop software for their solution and benefitted from web 2.0 technologies during the process. This process was as follows:

  1. 1.

    Build Readiness: Ill-structured problems were written and organized according to the experts’ views. The ill-structured problems were prepared based on designing and developing a computer program project. Then an orientation program which was about the introduction of the learning environment, necessary technologies, and how to use these technologies was prepared for the experimental groups and carried out by the researcher for three weeks. At the end of the program, the students were oriented regarding how to act in this learning environment.

  2. 2.

    Form and Norm Groups: Collaborative learning groups were formed in the experimental groups. While forming groups, the students’ level of programming skills was taken into consideration. The students’ level of programming knowledge was grouped as low, medium, and high based on students’ scores of the Basics of Programming course. Then 3 to 5 students at each level were selected to create heterogeneous groups. Thus more successful students in each group can help other students in different levels to gain programming knowledge. Although some of the studies claim that homogenous groups were more successful in computer programming (Hanks et al. 2011; Wu et al. 2019), other studies found that (Demir and Seferoglu 2020; Plonka et al. 2015; Zhang et al. 2016) heterogeneous groups were more successful in computer programming. But as for the educational perspective collaborative learning groups should be heterogonous (Johnson and Johnson 1999). This way of the forming group is based on the fact that collaborative problem solving is useful in terms of collaborative working, supporting the knowledge construction of group members, and developing better learning skills (Chorfi et al. 2020; Hung et al. 2008) In the EG1, virtual groups on Edmodo were created by the researchers for students to collaborate and interact with their peers and instructor while students in the EG2 communicated and collaborated with their peers and instructor face-to-face.

  3. 3.

    Determine a Preliminary Problem Definition: The virtual collaborative groups conducted meetings with Google Hangouts, Mind42 tool, and other Google tools to solve the ill-structured problem in the EG1. During these meetings, the groups defined the problem, proposed a solution, and developed a plan for solving the ill-structured problem. Then they took an action for the solution by searching for resources, necessary tools, and similar solutions. In the EG2, collaborative learning groups did similar activities using desktop software and organizing face-to-face meetings.

  4. 4.

    Define and Assign Roles: Students took an active role in the problem solving process. For example, one of the students was a project manager, another one was a follower of the Edmodo postings, the other one was a writer of the weekly report, and all of the students were programmers in the problem solving process. Therefore, to solve the ill-structured problem each student carried out the tasks in the experimental groups.

  5. 5.

    Engage in an Iterative Collaborative Problem-Solving Process: In order to solve the ill-structured problems, students did the following weekly activities with group members in the EG1:

    • Activities on Google tools

      1. i.

        Online meetings with groupmates on Google Hangouts: Each group in the EG1 conducted online meetings on Google Hangouts for the development of programming projects. In these meetings, one of the group members opened the project on Google Drive and share the screen with other group members, and coded together. Thus, they did the coding activities synchronously.

      2. ii.

        Online meeting(s) with the instructor on Google Hangouts

      3. iii.

        Preparing weekly report of online meetings with Google Docs

      4. iv.

        Sharing the computer program projects with groupmates and instructor on Google Drive: Each group member and the instructor can access the project at anytime and anywhere. In addition to the synchronous working on Google Hangouts, they could carry on asynchronously studying the programming project.

    • Activities on Edmodo

      1. i.

        Sharing reports of the weekly meeting

      2. ii.

        Making comments to other groups’ computer program projects

      3. iii.

        Following the news about computer program projects

        Screenshots of the web 2.0 technologies used in this study were presented in Fig. 1 as follows:

        Fig. 1
        figure 1

        Screenshots of the Web 2.0 technologies

        Full size image

        In the EG2, collaborative learning groups did similar activities weekly using desktop software (word processor software, drawing tools, presentation software) and communicating with group members face to face. In addition, group members shared the tasks related to the project and did the code activities on the computer or laptop weekly. Then each student presented and explained what he/she did for the project during weekly face to face meetings. Thus, all students learned programming together and made contributions to the project.

  6. 6.

    Finalize the Solution or Project: The collaborative learning groups in the EG1 shared the computer program project they developed with the instructor and the other groups on Edmodo while, in the EG2, students shared the software on their computers. Then all groups and the instructor gave feedback to the groups about their software project.

  7. 7.

    Synthesize and Reflect: All collaborative learning groups wrote a reflective report about the experiences they gained during the learning process.

  8. 8.

    Assess Products and Processes: The researchers assessed the computer program projects of all groups and the learning process.

  9. 9.

    Provide Closure: The software projects were stored in Google Drive and submitted to the instructor via Google Drive in the EG1, while in the EG2, the software projects were stored in a flash disk and shared with the instructor via the flash disk. In this way, the collaborative learning process was accomplished.

3.5 Implementation process

This study was carried out for eight weeks in the Object-Oriented Programming I-II courses. This study was conducted with two experimental groups and a comparison group. In this regard, some of the ethical issues were considered. All students were informed about the experimental process such as course content, learning environment, technologies that they use, and other requirements. They were asked to participate in this study as participation in this research was on a voluntary basis. Finally, the use of information collected from them for the research purpose was explained before the experiment.

Before the experiment started, firstly preparations such as building groups, presentation of the learning environment and technologies, preparing the ill-structured problems were done. Then the achievement test was applied to the students. After that, the implementation process was started and lasted eight weeks. The implementation process is shown in Fig. 2 below.

Fig. 2
figure 2

Implementation process

Full size image

During the implementation process of the study, the students in the experimental groups tried to solve ill-structured problems, studied these problems, and finally developed a computer program project. The instructor in the experimental groups helped students to learn the programming knowledge to solve the problem in the face to face session of the course. The researcher helped students to accommodate the learning environment presenting the features of the method, problem solving stages, technologies to be used, and other rules. Moreover, the researcher guided students during the problem solving stages such as defining the problem, asking questions to the solve problem, collaborating with groupmates, choosing the solution, and coding. In this respect, this kind of role of the researcher was scaffolding (Jonassen 1999). These scaffolding activities were planned and evaluated by the expert. During the experimental process, the researcher asked for advice on how to support students. Thus it was ensured that the researcher did not influence the experimental process.

Students did weekly activities based on scheduled problem solving stages in Fig. 2. These activities were presented in the development of the learning environment were done by students in the EG1 via web 2.0 technologies. However, the students in the EG2 worked with desktop software and communicate with group members face to face to solve ill-structured problems. At the end of the experiment, EGs took the achievement test and engagement scale. Additionally, their projects were assessed by the researcher and instructor with the project evaluation form. The students in the comparison group were taught the course with instructor-led methods such as lecturing, and demonstration. The instructor generally provides knowledge and this knowledge is given to the students via lecturing. Thus this approach used in the comparison group was a teacher-oriented method. In the comparison group, the instructor lectured the course content and demonstrated how to write a computer program, and the students did these same activities. They only worked to solve structured problems that the instructor created each week for the related programming topic.

3.6 Data analysis

This study employed One Way Analysis of the Covariance (ANCOVA) and One Way Analysis of the Variance (ANOVA) to answer the research questions. In order to find out whether students’ academic achievement scores differed across the treatment conditions, the ANCOVA test was conducted. Students’ pretest scores were taken as covariates. The dependent variable was the academic achievement and the independent variable was the treatment condition. The ANCOVA is a powerful statistics to reveal the treatment effect with reducing error variance and when groups start equal or not equal on covariate (Stevens 2009). On the other hand, to find out whether students’ academic engagement scores differed across the treatment conditions, the ANOVA test was conducted.

4 Results

4.1 Results related to students’ academic achievement

The ANCOVA test was conducted to examine whether there was any significant difference in terms of academic achievement between the experimental groups and the comparison group. The assumptions of the ANCOVA, normality, linearity, homogeneity of variances, and homogeneity of regression slopes were tested. It was found that there was no violation of the assumptions. The posttest achievement test scores were used as academic achievement scores in the analysis. The results of the analysis were presented in Table 3.

Table 3 Results of the ANCOVA test for students’ posttest achievement scores
Full size table

After adjusting for pretest scores, there was a significant difference on posttest achievement scores between the groups, F(2, 90) = 11.016, p < 0.001. The effect size was calculated as 0.197. It is considered to be a large effect (Green and Salkind 2004). As a next step, post-hoc comparisons with Bonferroni correction were performed for multiple comparisons to reveal which groups differed significantly in terms of their posttest achievement scores (Table 4).

Table 4 Means and adjusted means regarding posttest scores of students’ achievement scores using pretest achievement scores as covariate
Full size table

The results showed that there was a significant difference between the comparison group and both experimental groups. According to Table 4, the adjusted mean scores indicated that the EG2 had the highest adjusted mean score (M = 60.22). However, the adjusted mean scores of the EG1 and CG were 60.01 and 46.36, respectively.

In addition, during the experimental process, students in the experimental groups were exposed to different learning environments. Therefore, the ANCOVA test was conducted to whether there was any significant difference in terms of academic achievement according to their project scores and achievement test. To calculate the academic achievement scores, 50% of the project evaluation scores and posttest achievement scores were consolidated. Before employing the ANCOVA, the assumptions of the ANCOVA, normality, linearity, homogeneity of variances, and homogeneity of regression slopes were tested. It was found that there was no violation of the assumptions. The results of the analysis were presented in Table 5.

Table 5 Results of the ANCOVA test for EG students’ project and posttest achievement scores
Full size table

After adjusting for pretest scores, there was no significant difference found on the project and posttest achievement scores between experimental groups, F(1, 69) = 0.020, p > 0.05.

4.2 Results related to students’ academic engagement

The ANOVA test was conducted to examine whether there was any significant difference in terms of academic engagement between the experimental groups and the comparison group. The academic engagement had two subfactors, active learning, and collaborative learning; therefore, the ANOVA test was conducted for these two subfactors, respectively. First of all, the ANOVA was conducted to test whether there was any significant difference in terms of active learning engagement between the experimental groups and the comparison group. The results of the analysis were given in Table 6.

Table 6 Results of the ANOVA test for students’ active learning engagement scores
Full size table

The analysis results showed that there was a significant difference between the comparison group and experimental groups F(2, 91) = 4.38, p < 0.05, with an effect size of η2 = 0.088 which is considered to be a medium effect (Green and Salkind 2004). Before employing post-hoc comparison to determine which groups differ, homogeneity of variances was tested, and it was found that homogeneity of variances was not significant p < 0.05. Therefore, Dunnett’s C test was used to compare the groups. The results showed that there was a significant difference between the experimental groups. The mean scores were 25.41 for the EG1, 23.76 for the CG, and 20.63 for the EG2 (see Table 7).

Table 7 Means and standard deviations of active learning engagement scores
Full size table

Secondly, the ANOVA was conducted to test whether there was any significant difference in terms of collaborative learning engagement between the experimental groups and the comparison group. The result of the analysis was presented in Table 8.

Table 8 Results of the ANOVA test for students’ collaborative learning engagement scores
Full size table

The analysis results showed that there was a significant difference between the comparison and experimental groups F(2, 91) = 4.10, p < 0.05, η2 = 0.083 which was considered to be a medium effect (Green and Salkind 2004). Before employing post-hoc comparison to determine which groups differ, homogeneity of variances was tested, and it was found that homogeneity of variances was not significant p < 0.05. Therefore, Dunnett’s C test was used to compare the groups. However, pairwise comparisons with Dunnett’s C test did not show the difference between groups. Therefore, another method to reveal which groups differed significantly is the Contrast test method (Field, 2009). Firstly, the experimental groups considered as one group were compared to the comparison group. Then the experimental groups were compared to each other. Test results showed that there was a significant difference between the experimental groups and comparison group, t(29) = 2.21, p < 0.05. But, there was no significant difference between the experimental groups, t(66) = -0.67, p > 0.05. The mean scores were 21.24 for the EG2, 20.88 for the EG1, and 19.28 for the CG (see Table 9).

Table 9 Means and standard deviations of collaborative learning engagement scores
Full size table

5 Discussion

The purpose of this study was to investigate the effect of the learning environment designed based on the collaborative problem solving method and supported with web 2.0 technologies on students’ academic achievement and engagement. According to the results, there was a significant difference between the experimental groups and the comparison group in terms of achievement and collaborative learning engagement while there was a significant difference between the experimental groups in terms of active learning engagement.

First of all, the results indicated that there was a significant difference between the experimental groups and the comparison group in terms of achievement. This can be explained as a problem based collaborative learning environment had a significant effect on students’ course achievement than the traditional learning environment. The reason why the students were more successful in the experimental groups can be explained with the advantages of the collaborative problem solving method.

Firstly, students engage in an ill-structured problem solving process in this learning environment (Hmelo-Silver 2004; Hung et al. 2008; Marra et al. 2014). When students work on ill-structured problems which can be presented as in the real-life, they can outperform better than taking the content directly from instructors via lecturing (Balendran and John 2017; Korucu and Cakir 2018; Podges et al. 2014). The students in the experimental groups analyzed the ill-structured problems, searched for solutions, collaborated with group members, proposed solutions and decided on the best solution to solve the problem, and developed the solution by writing a computer program. During this collaborative problem solving process, the students were responsible for and regulated their learning process. Therefore, students outperformed better than students in the traditional learning environment.

Secondly, another advantage of the collaborative problem solving method is that students can work in groups and discuss different ideas to solve ill-structured problems (Duffy and Cunningham 1996; Hmelo-Silver 2004; Nelson 1999). Some studies showed that the effect of collaboration during the problem solving process enhanced students’ success compared to the traditional learning environment (Horak and Galluzzo 2017; Hou et al, 2016; Lee et al. 2017; Roberts 2017). In this study, the students worked and interacted with group members and other groups from defining the problem to solving it. In this regard, the students worked to solve an ill-structured problem according to the principles of the collaborative problem solving method by collaborating, interacting with resources, studying the course material, discussing ideas with group members and other groups, and communicating with the instructor. However, the students in the comparison group were taught with traditional methods such as lecturing and demonstration in lab sessions. For this reason, the students in the experimental groups had higher achievement scores than the students in the comparison group. Moreover, review studies about the effect of collaborative and problem based learning on achievement showed that this method increased students’ achievement (Dochy et al. 2003; Wilder 2015). In this regard, this study contributed to the literature by stating that using collaborative problem solving method can enhance students’ achievement than traditional methods in the community college context.

In terms of project scores and posttest achievement scores, the results showed that no significant difference was found between the experimental groups. The same method, the collaborative problem solving method, was used in the experimental groups. However, web 2.0 technologies were used by the students for communicating, collaborating, and interacting with others to solve problems in the EG1 while face-to-face communication and desktop software were used by the students to solve problems in the EG2. Therefore, web 2.0 technologies did not affect students’ success. Although this result lends support to previous findings in the literature (Dennis 2003; Şendağ and Odabaşı 2009), it does not concur well with others (Hou et al. 2016; Korucu and Cakir 2018). The result of this study can be interpreted as the collaborative problem solving method is a student-centered method where the students regulate their learning process, try to solve the problem and the instructors’ role is guidance. (Marra et al. 2014; Nelson 1999). Therefore, two experimental groups performed similarly in the course. The only difference between the groups was the type of technology for supporting the learning process. This result is consistent with Clark’s view. Clark (1994, p. 27) indicated that “Media and their attributes influence the cost or speed of learning”. The method used in these two learning environments was based on constructivism in nature. Therefore, different technology support in these learning environments did not influence achievement significantly.

The second main result of this study was about academic engagement. The academic engagement was examined in two subfactors as active learning and collaborative learning. In terms of active learning engagement, the result indicated that there was a significant difference between the experimental groups. Active learning engagement scores of the EG1 were higher than the EG2. This result can be interpreted as the students in the EG1 engaged in problem solving much more than the students in the EG2. As indicated before, the only difference between the two experimental groups was the type of technology to support collaboration and problem solving. Characteristics of web 2.0 technologies as usefulness, communication facilities, easy access, and flexibility in regard to time can affect students’ active learning engagement. Previous research reported the positive relationship between the use of technology in the learning process and students’ engagement level (Chen et al. 2010; Laird and Kuh 2005). In this regard, the students in the EG1 communicated with group members, namely other students, and instructor, had online meetings, and participated in the learning process outside the school from everywhere at any time actively via web 2.0 technologies. As web 2.0 technologies offer these opportunities, the students in the EG1 might participate in the learning process actively. However, it was found that students’ active learning engagement scores in the comparison group did not significantly differ between groups. This result is inconsistent with some studies in the literature (Alioon and Delialioğlu 2017; Lavonen et al. 2002; Marra et al. 2014). According to the literature, learning programming is a difficult area for students (Bravo et al. 2013; Cheah 2020). Therefore students in the comparison group felt that the programming courses were challenging for them. Since students should engage in learning programming continuously and show their persistence to learn the programming (Cheah 2020), they worked individually and engaged in instructional activities during the learning process.

The other result about academic engagement indicated that the students’ level of collaborative learning engagement in the experimental groups was significantly higher than the students’ in the comparison group. Similar results found in literature support this finding (Alioon and Delialioğlu 2017; Dabbagh et al. 2000; Malhiwsky 2010). This result can be explained with the principles of constructivist learning stating that knowledge is constructed through social negotiation (Duffy and Cunningham 1996; Fosnot and Perry 2005). In other words, learning is a process of construction of a learner’s interactions with other students, instructors, and environment (Jonassen 1999). In the current study, students in the experimental groups had meetings with group members, discussed different ideas with group members, wrote a report about discussed ideas, gave feedback to other groups, and communicated with the instructor to solve the problem. However, the students in the comparison group wrote and applied the same programming codes to their applications which were taught by their instructor. In this regard, their level of collaborative learning engagement was lower than the experimental groups. However, the result showed that the experimental groups’ level of collaborative learning engagement did not significantly differ. This result can be interpreted as the use of web 2.0 technologies or desktop software and face to face communication to support collaborative problem solving did not affect students’ level of collaborative learning engagement. In other words, the experimental groups did the same activities such as discussion, collaboration, interaction with groups via web 2.0 technologies or desktop software, and face to face communication to solve the problem and to learn the course content. In this regard, the students in the experimental groups reached a similar level of collaborative learning engagement.

5.1 Implications

Based on the results, a set of implications were derived. First, this study provides evidence that the collaborative problem solving method is effective to enhance students’ achievement in community college. Thus this method may be utilized at community colleges to increase students’ academic achievement. Second, integrating web 2.0 technologies into the learning environment where collaborative problem solving is used can be beneficial in terms of motivating students actively to the learning process. For example, videoconferencing tools can provide online meetings for group work, social networking tools can be useful in terms of creating online communities, sharing and discussing group working, file sharing and storage tools can make easier online sharing of group files and mindmapping tools can help to organize ideas about group work. Although Google tools, Mind42 tool, and Edmodo are used in the current study, many tools can be selected for similar aims according to the context. Third, collaborative learning activities can be employed in the learning environments at community colleges due to the positive effect of the collaborative problem solving method on students’ level of collaborative learning engagement.

5.2 Limitations and future directions

As in any social studies, this study is bound with some limitations. The first limitation is related to sample size. Therefore, similar studies should be conducted with larger sample sizes in different contexts to generalize the results. Second, due to the limited time for the implementation of the study, the retention test cannot be administrated to the groups. A retention test can be administered to examine the retention level of students in groups after a certain period of the experiment end time in future studies. Third, although researchers took the preventive measures to minimize their influence on the study, due to the fact that the researchers provided orientation for the collaborative problem solving process and web 2.0 tools that support collaborative problem solving process, the effect of their involvement on research results were not measured and it may be seen as a limitation. Forth, this study was limited with the opportunities that Google tools, Mind42 tool, and Edmodo provided. Future studies can focus on other tools that have more learning opportunities.

6 Conclusions

This study made an important theoretical contribution to the literature by indicating the positive effect of collaborative problem solving on students’ achievements and collaborative learning engagement levels. Furthermore, this study proved that web 2.0 technologies increased students’ active learning engagement levels while they had no impact on students’ collaborative learning engagement levels. Thus, using collaborative problem solving method and web 2.0 technologies may be useful to develop students’ knowledge and skills in their courses. Accordingly, they can be equipped with the competencies needed in the business sector after they graduated. On the other hand, this study clarified and achieved the design issues of collaborative problem solving method at the community colleges. In this respect, this study can provide a framework to enhance the quality of the learning environments at community colleges.