Introduction

Within science education, it is critical to help students learn to search for, critically evaluate, and use information to solve problems (Sinatra, Kienhues, & Hofer, 2014). One way to do so is to allow authentic problems, rather than a list of topics, drive learning (Gijbels, Dochy, Van den Bossche, & Segers, 2005). Problem-based learning (PBL) is one such instructional model that requires students to address authentic, ill-structured problems through scientific inquiry and collaborative learning (Malmia et al., 2019; Savery, 2019). Meta-analytic results have indicated that PBL is more effective than lecture at promoting principles and application-level learning outcomes (Gijbels et al., 2005). PBL is particularly successful when paired with computer-based scaffolding, which can provide conceptual, strategic, and metacognitive support to students as they engage in problem-solving (Ertmer & Glazewski, 2019).

Key competencies for success in PBL include (a) advanced information literacy, defined as the ability to search for, evaluate, and validate information to support the production of solutions (American Library Association, 2000; Diekema, Holliday, & Leary, 2011) and (b) strong argumentation skills, defined as the ability to make propositions based on evidence that supports these solutions (Kim et al., 2020; Toulmin, 1958). K-12 students who have only experienced teacher-led direct instruction often struggle within PBL, both due to inadequate information literacy and argumentation skills, as well as their lack of experience at self-directed learning (Buchanan, Harlan, Bruce, & Edwards, 2016). The key to providing an engaging PBL experience is posing a rich problem; this can often be enhanced by inviting students to be directly involved in the context in which the problem occurs (Parong & Mayer, 2018). Having students visit these authentic contexts is ideal; however, it is not always possible. A viable alternative is the use of virtual field trips (VFTs), which grant learners opportunities to experience authentic scientific situations without ever leaving the classroom (Mead et al., 2019).

Computer-based scaffolding has been shown to enhance students’ information literacy (Glazewski & Hmelo-Silver, 2019) and argumentation skills (Wallon, Jasti, Lauren, & Hug, 2018). Nonetheless, few studies utilize both computer-based scaffolding and VFTs to improve learners’ information literacy and argumentation skills within the context of PBL for science education, which begs the question of whether similar positive effects would occur in such a context.

Literature Review

PBL in Science Education

Learner-centered environments allow students to explore information, discuss, and collaborate with classmates while solving problems and proposing evidence-based explanations for scientific phenomena (Rockman, 2019). Producing evidence-based explanations requires that students (a) evaluate sources, (b) accurately describe gathered information, and (c) justify decisions based on newly acquired knowledge (Belland, Weiss, Kim, Piland, & Gu, 2019; Kim, Belland, & Axelrod, 2018b; National Research Council [NRC], 2012). PBL can afford an ideal educational environment for science learning, where students can solve authentic, ill-structured problems through active argumentation and the creation of evidence-based claims (Merritt, Lee, Rillero, & Kinach, 2017). Savery (2019) described four characteristics of PBL:

1. Lessons begin with the presentation of authentic, practical problems, which in themselves are tools for learners to obtain knowledge.

2. The problems must be ill-structured; that is, defined as problems that (a) lack a full description of problem elements and how they interact, (b) enable various approaches for problem-solving, and (c) allow for the definition of problems to change as more information is gathered.

3. Learning should be self-directed by students, while teachers should stimulate higher-order thinking and encourage students to solve problems based on their prior experiences.

4. Team members present options and reach a consensus on the final solution.

Moreover, field trips have been prevalently used to enhance students’ contextualized experiences by enabling access to representations of and various perspectives on natural phenomena within PBL (Cheng & Tsai, 2019). These experiences in authentic settings have been demonstrated effective in improving students’ achievement, recall of scientific knowledge, and attitudes towards science (Choi, Kim, & Choe, 2018). However, the negative implications associated with field trips (i.e. safety challenges, financial burdens, and time constraints) call for a more feasible alternative (Mead et al., 2019).

Virtual Field Trips in PBL

VFTs, defined as computer-based environments that allow students to virtually visit a place without ever leaving the classroom (Mead et al., 2019), are suggested as a contemporary solution to address the challenges posed by field trips. Notably, they can facilitate students’ interaction in learning communities, provide multimedia resources as contextualized learning materials, and offer great insight into the diversity of virtual landscapes (Minocha, Tilling, & Tudor, 2018). When embedded within PBL environments, VFTs allow students to solve scientific problems through hands-on learning, which can enhance student exploration and constructive learning (Banic & Gamboa, 2019). VFTs are especially applicable to earth and environmental science learning as they can generate three-dimensional features of planet Earth, which may sharpen students’ spatial thinking skills and the ability to visualize and understand environmental and geographical problems (Klippel et al., 2019).

Combined Effects of Information Literacy and Argumentation Skills in PBL

The two main phases that frame the classroom implementation of PBL are problem discovery and problem management (Barrows & Tamblyn, 1980). Information literacy is essential during problem discovery, since it enables students to locate reliable sources and validate them for use in the next phase. Afterwards, argumentation skills become vital during problem management, a phase in which students check, synthesize, and link gathered evidence to make claims that address the posed problems. It is crucial in PBL that students have a systematic process to identify what they know and what they need to learn in order to develop solutions (Belland, Walker, & Kim, 2017). Therefore, learners using PBL should have enhanced information literacy to be able to reorganize information, synthesize it into knowledge, and leverage it into practice (Moselen & Wang, 2014). Poor information literacy may hinder students from finding and utilizing information online due to an overwhelming number of sources (Manny & Ellis, 2019). Students with strong information literacy and argumentation skills can successfully employ scientific content knowledge in support of their claims (Cardetti & LeMay, 2019). Due to the scarcity of studies considering the combined effects of information literacy and argumentation skills in PBL, it is unclear how to balance and emphasize these two during each step of the learning process. Taking into account that the importance of information literacy is often underestimated when compared with argumentation skills in PBL (Macklin, 2001), addressing this issue in future studies becomes vital.

Scaffolding for the Improvement of Information Literacy and Argumentation Skills

Scaffolding was initially established to support children or novice students performing tasks otherwise not possible (Wood, Bruner, & Ross, 1976), but recently, it has been shown to benefit learners as they progress through each of the problem-solving steps in PBL (Ertmer & Glazewski, 2019). Advances in technology have allowed scaffolding to be provided by computer systems. When computer-based scaffolds are incorporated to improve information literacy and argumentation skills, they can provide (a) conceptual scaffolding in the form of dynamic feedback that helps students identify needed information for problem-solving (Turner, Fuchs, & Todman, 2015); (b) strategic scaffolding through questions and prompting, which can help students search for, validate, and utilize information by bootstrapping effective search strategies (Wiley et al., 2009); and (c) metacognitive scaffolding, which supports reflection and reasoning skills as students interact with others (Kramarski & Gutman, 2006). Using different types of scaffolding to support argumentation can help students generate varied hypotheses along with their justifications, which benefits the overall problem-solving process (Noroozi, Kirschner, Biemans, & Mulder, 2018). Furthermore, recent meta-analyses (Kim, Belland, & Walker, 2018a) have found both positive and negative effects of computer-based scaffolding types vis-à-vis changes in higher-order thinking skills and cognitive outcomes in PBL. For example, conceptual scaffolding was less effective than other types of scaffolding on improving students’ cognitive outcomes, and negative results have been found in some studies, particularly in science education (Kim et al., 2018a). Possible reasons are due to some students’ adverse attitudes towards technology use (Rashid & Asghar, 2016) and insufficient provision of conceptual support on what should be considered for scientific problem-solving (Kim et al., 2018b). Conversely, metacognitive and strategic scaffolding showed notably positive effects in enhancing students’ self-evaluative process to solve ill-structured problems, and self-regulative strategy to reflect on their knowledge construction process (Kim et al., 2018a). This dichotomy depended upon how scaffolding was implemented. Few studies included in these analyses investigated how and which types of scaffolding promote information literacy and argumentation skills in PBL. Thus, this study aimed to add to the literature by investigating the positive and negative effects of different types of computer-based scaffolding on information literacy and argumentation skills within PBL integrating VFTs. Accordingly, this study addresses the following research questions: (1) does scaffolding impact students’ information literacy and argumentation skills during the specific problem-solving process in PBL with VFTs? (2) How do the different types of scaffolding positively or negatively affect students’ outcomes in (a) information literacy and (b) argumentation skills for scientific problem-solving?

Method

Participants

Participants were 29 students enrolled in a credit recovery course on environmental science in a public high school in the Intermountain West of the United States. Credit recovery courses are offered during the summer, allowing students who failed a core class in the academic year to retake the course, thereby earning credit. Nine students had just completed 10th grade, while 20 students had just completed 11th grade. Seventeen (58%) were female, while twelve (42%) were male. The course instructor had 20 years of experience teaching high-school science using a PBL approach. The PBL environment afforded discussion and argumentation from unique stakeholder perspectives; therefore, participants were assigned into groups of 3–4 students to address the central problem (management of air quality within the local mountain valley) from the perspective of a different stakeholder (e.g. environmentalists, common citizens, State government officials, asthma sufferers, and meteorologists). That is, adopting a jig-saw approach, different stakeholder roles were assigned to each member in the same group.

Research Design and Sampling Strategy

We used an explanatory sequential mixed methods design consisting of two phases: quantitative analysis, followed by a qualitative case study (Creswell, 2003). Statistical data obtained from 29 students’ information literacy and argumentation skills pretests and posttests were analyzed to determine the effects of scaffolding in PBL. Then, following a case study approach, we analyzed in-depth qualitative data to examine the positive and negative results yielded from the quantitative analysis (Ivankova, Creswell, & Stick, 2006). Next, we collected and analyzed interview and observation data from a purposeful sample of 14 students whose information literacy and argumentation skills scores increased or decreased. Students who scored at or below the 25th and at or above the 75th percentiles on the pretest-posttest differences on the information literacy and argumentation skills tests were identified and assigned to four groups (Table 1).

Table 1 Group composition for case study
Full size table

Overview of PBL Process: Air-Quality Issue in Earth Science Lessons

This study was embedded in the context of two computer programs (i.e. VFTs and the Connection Log) addressing air-quality issues (Fig. 1). The content and support in these programs were developed based on State Core Standards for Earth Science, (https://www.uen.org/core/core.do?courseNum=3600) which emphasize (a) hands-on experience with scientific concepts and phenomena, (b) collection and accurate interpretation of contextual information, and (c) use of a scientific inquiry process for building solutions. All learning activities were validated by one instructional design professor and an earth science secondary teacher through common agreement. The VFTs were designed to enhance students’ information literacy, while the Connection Log was designed to improve their argumentation skills. The next sections describe the types of scaffolding interventions within the two computer programs in more detail.

Fig. 1
figure 1

Overall process for solving air quality issues in PBL

Full size image

Virtual Field Trips.

VFTs (www.sakura7k.dothome.co.kr/xe/) were used to allow students to virtually collect air-quality data in different locations and consider the reasons for variation in air quality. Students could request data on air quality whenever needed, and real-time information was given. During the first 2 days of the PBL unit (Fig. 2), students performed tasks related to three main objectives: (a) developing a topic: the teacher introduced the ill-structured/authentic problem and invited the students to devise multiple solutions to ameliorate the bad air quality in their city; (b) developing a search strategy: students were introduced to the relevant vocabulary and a study guide for air quality, followed by a brief animated tutorial (“HELIOS”: helios.weber.edu) that taught them how to identify and locate needed information; and (c) identifying potential sources: the VFTs provided students with lists of online resources and reading materials related to air quality for students to examine. Days 3–5 of the PBL unit focused on (d) searching and evaluating sources of information: students used the VFTs to explore air quality in different cities and analyze their surrounding topography, related facilities, and natural environments. The VFTs also allowed students to compare the air quality from several cities with specific air-quality data (e.g. ozone, CO2, NO2, PM2.5, and PM10). For example, students measured air quality in Houston, famous for oil refineries that contribute to air pollution. Afterwards, they could choose another city with a similar air pollution index (API) to that of Houston, but having different component values within the API. This process facilitated student understanding of air-quality issues. The final objective, (e) recognizing how to use the information, had students working in groups within the VFTs to solve these problems. A unique stakeholder role was assigned to each member in the group (e.g. environmentalists, common citizens, etc.) so that groups could create arguments about air-quality issues from the perspective of different stakeholders. Three types of scaffolding (i.e. conceptual, metacognitive, and strategic) were offered to students performing tasks in the VFTs. These scaffolding types supported students at cognitive and metacognitive levels, according to their needs and learning status. All computer-based scaffolds were provided in the form of pop-up windows taking place in three diverse settings: (a) students’ requests, by clicking the “ask for help” button; (b) fixed-time intervals; and (c) when students finished each task. Although students received different types of scaffolding according to their learning progress, the specific supports for each type of scaffolding were identical for all students (for examples of scaffolding types used in each VFT task, see ESM A).

Fig. 2
figure 2

Air quality problem in virtual field trips

Full size image

Connection Log.

This program supplemented the initial understandings of the air-quality issues in the VFTs by offering students the opportunity to carry out PBL online in order to build evidence-based arguments to solve problems. Students used this program during days 6–11. They moved through five different tasks:

1. Define the problem from their stakeholder perspective.

2. Determine needed information.

3. Organize needed information.

4. Develop claims about the causes of bad air quality and why their solutions will address such. 5. Link evidence to their claims (Fig. 3).

Fig. 3
figure 3

Connection Log

Full size image

Students were supported by strategic and metacognitive scaffolding. Strategic scaffolding presented students with strategies to organize gathered information and use it as evidence for the development of problem-solving claims. Metacognitive scaffolding guided students as they organized their thoughts and information and offered prompts that encouraged them to reflect upon their learning process. Further, whenever students finished tasks, metacognitive scaffolding prompts asked learners to analyze their work from the perspective of other stakeholders and then agree upon a decision to solve the problem. This process helped students deliberate on the learning goal, learning process, evidence, and arguments. All students received the same type of scaffolding as their learning progressed.

In Class.

At the end of the unit (day 12), all groups presented their solutions and associated arguments in short, 10-min presentations, in which each group shared their screen of the Connection Log. A Q&A session took place after each presentation, allowing for other groups and the teacher to evaluate the presenters’ outcomes and performance. Reflecting on the presentations and the feedback afforded improvement of students’ learning and problem-solving process.

Data Collection

Information Literacy Test.

This study employed two versions of the Tools for Real-Time Assessment of Information Literacy Skills (TRAILS) to assess high-school students’ information literacy as the pretest and posttest (Kovalik, Yutzey, & Piazza, 2012). The items (ESM B) were created based on information literacy standards from the American Association of School Librarians and the Ohio Academic Content Standards (Voelker, Schloman, & Gedeon, 2013). The 25 multiple-choice items in the pretest and posttest measure students’ abilities to (i) develop a topic (5 items); (ii) develop, use, and revise a search strategy (4 items); (iii) evaluate sources and information (4 items); (iv) identify potential sources (6 items); and (v) recognize how to use the information responsibly, ethically, and legally (6 items) (Schloman & Gedeon, 2007). Five school librarians and ten 12th grade teachers positively rated the face validity of the TRAILS based on the extent to which the items can actually measure students’ information literacy (Salem Jr., 2014). The results indicated that 79.4% of librarians and teachers reported that the items were valid assessments, while 15.57% of librarians and teachers reported that the items were valid with revision. Furthermore, several empirical studies demonstrated high reliability of TRAILS test scores (Cronbach’s alpha from 0.81 to 0.82) among high-school students (Arnone, Small, & Reynolds, 2010). However, the reliability of the TRAILS was not calculated in this study due to the small number of participants (n = 29), since having fewer than 30 participants can lead to unstable component patterns (Yurdugül, 2008).

Argumentation Skills Test.

Students took an argumentation skills test before (pretest) and after (posttest) the PBL unit (ESM C). The test required students to (a) read a paragraph on the use of electric vehicles, and (b) make a claim on how to solve air pollution issues, providing evidence from the paragraph. The paragraph content included all six components (i.e. claim, data, warrant, backing, qualifier, and rebuttal), which are indicators of good argumentation (Toulmin, 1958). The validity of the reading material was established by three experts: one instructional design professor, one science education professor, and one high-school science teacher. Argument quality was assessed using a rubric developed by McNeill, Lizotte, Krajcik, and Marx (2006), which assigned 0–2 points for the quality of each of the following elements: claims, evidence, and reasoning (for a total argument quality score of 0–6 points; ESM D). Previous research evaluating the construction of students’ scientific explanations revealed that the scores from this rubric yielded acceptable reliability (Cronbach’s alpha = 0.81; McNeill et al., 2006). Using McNeill et al.’s (2006) rubric, individual written reports were assessed by two independent raters, and Cohen’s kappa was used to measure inter-rater reliability. Results (kappa = 0.86 with p < .05) indicated a substantial agreement between raters’ scores.

Interview Protocol.

A 30-min semi-structured interview protocol was designed in accordance with the purpose of this study (ESM E). The interview questions consisted of three sections: section 1, Experience on searching for information; section 2, effects of VFTs on students’ information literacy; and section 3, experience on argumentation in PBL and effects of scaffolding. Depending on the participant’s response to the main question in each section, follow-up questions were asked to gain more in-depth information. All interviews were audio-recorded and transcribed.

Observations.

The study included two types of observations. First, qualitative subsamples of students’ classroom tasks within PBL were recorded through webcams installed on students’ personal computers and subsequently transcribed. Additionally, discourse taking place in small groups or between students and teachers (or researchers) was captured. Afterwards, all activities on students’ computers were recorded using Screencastify to observe students as they engaged with the VFTs and the Connection Log. Observation data were used to (a) identify students’ nonverbal expressions while using scaffolding, (b) determine learners’ unique problem-solving approach, and (c) examine their information literacy and argumentation skills during the performance of activities. The total length of the observed activities was 660 min.

Log Data.

Log files stored in the VFTs and the Connection Log were also utilized as a complement to interview and observation data. These files were generated automatically and saved to the server, from the time students logged into the VFTs and the Connection Log until they logged out. Log files included time (i.e. minutes) spent on individual and collaborative learning during each task of the two programs, what students wrote in response to scaffolding prompts, and the number of interactions with scaffolding. Thus, log data provided an additional perspective on the quality of students’ information literacy and argumentation skills, contributing to the trustworthiness of our interpretations.

Data Analysis

Quantitative Analysis for RQ 1

Given the small number of participants (n = 29), it was difficult to satisfy the assumption of a normal distribution. Therefore, the Wilcoxon signed-rank test was used to investigate whether the median difference between pairs of observations was zero (Sheskin, 2003). The information literacy and argumentation skills tests were at the interval level, which satisfied the assumption of non-parametric analysis (Sheskin, 2003).

Qualitative Analysis for RQ 2

All qualitative data were analyzed through the lens of phenomenography, which is used to explore the ways participants experience a certain phenomenon (Akerlind, 2005). This approach was chosen to characterize students’ positive or negative experiences with computer-based scaffolding in scientific problem-solving. The analysis involved the following steps: first, two coders examined interview transcripts and observation protocols to identify whether there were enough data to represent the impact of multiple sources of scaffolding on each student’s information literacy and argumentation skills. Then, a coding scheme was created, informed by theoretical literature and the factors/steps embedded in the VFTs and the Connection Log. The coding scheme and the initial codes were generated through a consensus of the two coders, whose inter-rater reliability (α = 0.82) was above the minimally acceptable level (α = 0.667, Krippendorff, 2004). There were 21 codes related to information literacy, 16 codes related to argumentation skills, and 13 codes for students’ experience in using scaffolding (ESM F). After generating a list of initial codes, axial coding was used to cluster individual codes into overarching categories. The fourth step was to use several review processes from multiple data sources (i.e. interview, observation, and log data) to confirm, replace, or terminate a category, as the qualitative results needed to be supported by interpretations of at least two evidence types. Following the finalization of categories, relationships between these categories were constructed based on similarities and differences. These relationships provided information on the effects that computer-based scaffolding had on learners’ information literacy and argumentation skills, which were further classified into positive and negative effects as informed by research by Bella (2017, 2019).

Results

Research Question 1

The analysis of quantitative data revealed that computer-based scaffolding was successful in increasing students’ information literacy scores, enhancing their ability to find, evaluate, and utilize information in the context of PBL. The scores of 20 students, 4 students, and 5 students increased, decreased, or stayed the same, respectively (Table 2). The data were not normally distributed, Shapiro-Wilk = 0.89, p < 0.01. The Wilcoxon signed-rank test indicated that the posttest median was statistically higher than the pretest median, p < 0.01, Cohen’s d = 0.37.

Table 2 Wilcoxon signed-rank test for information literacy
Full size table

In addition, 20 students’ argumentation skills posttest scores increased compared with their pretest scores (Table 3). In fact, results indicated that scaffolding improved most students’ argumentation skills, p < 0.01, Cohen’s d = 0.50.

Table 3 Wilcoxon signed-rank test for argumentation skills
Full size table

The Wilcoxon signed-rank tests for both the information literacy and argumentation skills measures indicated that the negative ranks performed better in posttests than in the pretests and were more prevalent than positive ranks. These results revealed that scaffolding significantly improved students’ information literacy and argumentation skills. However, scaffolding also had a negative or no impact on around 31% (n = 9) of participants’ information literacy and argumentation skills scores. All students (n = 20) whose scores increased between the information literacy pre- and posttest showed considerably better (n = 15) or somewhat better performance (n = 5) in the posttest than in the pretest for argumentation skills; however, two out of four students with negative results in the information literacy test obtained the same score in argumentation skills pre-posttest results. Five students did not have any changes between pre- and postscores in both tests.

Research Question 2

The second research question addressed positive and negative effects of different scaffolding types on students’ information literacy and argumentation skills in each task within the VFTs and the Connection Log. Table 4 provides an overview of the results.

Table 4 Overview of scaffolding intervention, results, students (group), and tasks
Full size table

Research Question 2a: Scaffolding and Information Literacy in VFTs.

Observations revealed that students who lacked experience searching for information initially utilized general terms that were too broad (e.g. air quality), resulting in an excessive amount of information and frustration. Once students received scaffolding, they showed an increased ability to use more specific terminology on their searches and to distinguish between reliable and unreliable sources.

Conceptual Scaffolding.

This type of scaffolding invited students to consider what to search for and where, with prompts such as “You finished your measurement. The next step is to figure out the national air quality standard for the pollutants. Check the following URL.”

Positive Results.

Conceptual scaffolding showed positive results as students performed the five tasks during the completion of their VFTs. Students were particularly successful when it came to developing a search strategy (i.e. task 2), evaluating sources of information (i.e. task 4), and recognizing how to use the information (i.e. task 5). Interviews indicated that such scaffolding supported locating air-quality data more easily and distinguishing between accurate and inaccurate information. To illustrate, Thomas commended the manner in which conceptual scaffolding helped him find information to understand science content and interpret air-quality data (i.e. task 2). Another student, David, found this type of scaffolding useful as he completed task 2:

David: When I searched for information ... The pop up explained the way to search for books. [on screen: “When you want to purchase a certain book, how did you search for this book? You probably put the title, publisher, year, authors of this book. Please consider this”]. This was helpful for me to use specific search terms for air quality information.”

Conceptual scaffolding also helped students approach the task from their assigned stakeholder perspectives (i.e. task 5). As an example, Elisa was assigned the role of an environmentalist. Although, at first, she expressed difficulty understanding the term stakeholder, scaffolding supported her in learning the new lexicon and integrating all the gathered information from an environmentalist’s point of view. Conceptual scaffolding led her to use search terms such as environmentalist and the natural environment. Recordings revealed that initially Elisa felt confused about conflicting information she had found, which made her struggle to validate her sources. However, after interacting with the scaffolding, which she often described as “wonderful” and “cool,” her frustration diminished. Elisa indicated satisfaction with the scaffolding support to help her determine the most valid and legitimate information among divergent online sources (i.e. task 4):

Elisa: I was embarrassed when I found two conflicting [pieces of] information. The software [VFTs] was cool, it gave me hints and feedback. It helped me realize which information was the most reasonable and validated. It didn’t just hand me the answers, it helped me to find the solution. My solution was to search information from websites with .gov, .org.

Negative Results.

Elliot had a negative experience when this type of scaffolding was provided to help him identify potential sources (i.e. task 3). He believed in the reliability of the material obtained with the support of VFTs’ conceptual scaffolding and thought that “all the information was important.” Consequently, he wanted to read through all the resources but was incapable of doing so due to the vast number of sources and the limited time to go over them. He struggled to “remove information sources and select the best ones to support [his] claims well.” In his case, too much information hindered his ability to address the PBL problem.

Metacognitive Scaffolding.

This type of scaffolding encouraged students to reflect on their learning and current understanding of topics, with prompts such as “In 2015, air quality in Mountain City rapidly got better and pollution—particularly in Ozone levels—decreased. What are possible reasons for this?”

Positive Results.

Metacognitive scaffolding showed positive results in supporting students as they worked on tasks 4 and 5. Specifically, the scaffolding in task 4 helped Julie realize what she knew and what she needed to know to come up with a creative solution for air-quality issues. Then, scaffolding during task 5 assisted her in rearranging information in order of importance and effectively using it as evidence to support her specific claims:

Julie: I was confused and did not know where to start. The questions do not have any correct answers. I don’t even know how I got out of my confusion, but computers helped a lot. I could deal with various sources about air quality. After gathering information, I just chose the top information and changed an unsatisfied thing into the down [referring to organizing information with the highest valued information at the top of her list and the least valued at the bottom]. After all, I can check all my information for problem-solving at a glance.

Brett also benefited from the metacognitive scaffolding provided during the completion of task 4, as it reinforced the information he gathered and assisted him in locating resources to interpret and use the air-quality data to solve problems. After Brett collected data indicating bad air quality in Orlando (FL), scaffolding offered information on the past 10 years of air-quality data for this city—one of the best in the nation in terms of air quality. While completing task 5, Brett examined the numbers and realized that he might have collected inaccurate or temporary data; he learned from online resources that a rocket had been launched from Kennedy Space Center (near Orlando) at the time of his measurements, which resulted in lower air-quality readings. In his interview, Brett discussed how the scaffolding enabled him to successfully locate, assess, and interpret the information he gathered for solving problems:

Brett: [VFTs] helped me to find good information. I used this software to find air data in Orlando. I thought the air quality in Orlando would be so bad and it was. But a pop-up window came on and it showed Orlando’s air quality history. Then I knew the air quality that I found was only temporary. I love this program, it helped me to find accurate air quality data. Without the pop-up window, I would’ve reported Orlando as the worst air quality city.

Collectively, the positive findings from the metacognitive scaffolding interventions appear to have increased students’ independence and self-confidence.

Negative Results.

Most negative experiences with the scaffolding seemed to occur when students were completing task 3 (i.e. identifying potential sources). A possible preference for faster means to finish tasks, which does not involve in-depth thinking, would account for this negative outcome. As an example, Elizabeth did not seem interested in using the scaffolding in the VFTs. She closed all the pop-ups as they appeared, suggesting that she did not want to use any of the recommended sites to look up information. Elizabeth chose to make Wikipedia her sole source of information, explaining that “everyone uses Wikipedia and it is an encyclopedia; all truth.”

Strategic Scaffolding.

This type of scaffolding guided students with next steps for solving problems, with prompts such as “Air quality can be determined by several levels of pollutants, and you need to check these pollutants, which have a huge impact on the city you selected. Please visit this website.”

Positive Results.

Strategic scaffolding played a key role during the learning process. It supported William when searching, validating, and utilizing information (i.e. tasks 2, 4, and 5). The next excerpt illustrates his experience in judging the credibility of the resources he located (i.e. task 4), adapting the terms used for his searches (i.e. task 5), and effectively reducing the overall number of results in a quick manner:

William: You know, the number of my first search results was about 20,900. I changed the search terms with help from [3-second pause]. How would I say this? [4-second pause]. OK, Computers. And I greatly reduced the results number. [2-second pause] Yeah!! Very helpful.

Negative Results.

The strategic scaffolding was a hindrance for some of the students. Screen recordings of Elizabeth’s interaction on the computer showed her typing statements such as “I hate this” and “so confusing,” which revealed Elizabeth’s dissatisfaction with the scaffolding. Moreover, her interview suggested a preference for teacher-led instruction:

Elizabeth: I think the learning was not efficient. The science class I took before, I did not have to gather information. My teacher would explain concepts and I just needed to correct the questions with the handouts. But if you Google air quality so many things come up and probably more than half of this information would be something that I do not need. There is no way I am going to look over all the information [2-second pause] it is not worth my time and, even if I find information I would not know if it is the right information to use [3-second pause] On the VFTs, things would pop up constantly and it was difficult for me to concentrate on my work. The pop-ups got in the way, so I just closed them all.

Likewise, Kevin did not benefit from the scaffolding in the VFTs. Analogous strategic scaffolds were provided regularly, and Kevin considered them too repetitive and inconvenient, which decreased his motivation to locate data. Consequently, he was reluctant to search for information on air quality across different cities using the VFTs and engaged in off-task behaviors.

Research Question 2b: Scaffolding and Argumentation Skills in PBL.

Two scaffolding types—metacognitive and strategic—were provided in the Connection Log to improve argumentation skills.

Metacognitive Scaffolding.

An analysis of observations and student interviews revealed that metacognitive scaffolding (i.e. reinforcing the need to evaluate what is happening, who it is affecting, and how it affects the stakeholders) encouraged students to find justifications that involved critical thinking skills and a more in-depth understanding of the given topic.

Positive Results.

Positive results were found vis-à-vis students’ completion of all five assigned tasks. To illustrate, Thomas was satisfied with the way in which the Connection Log inspired him to broaden his content knowledge about air quality and better enabled him to obtain more logical and reasonable conclusions from the data gathered online (i.e. task 4). Scaffolding also helped him to notice a different quality of outcomes depending on how well each task was completed, and to realize the importance of following scaffolding guidance to make stronger claims and draw well-founded conclusions. Another student, Elliot, benefited from the scaffolding by considering the problem from different perspectives (i.e. task 1) and selecting valid information for his claims (i.e. task 2). This, in turn, enabled him to develop a strong argument:

Elliot: Some of the biggest problems with air pollution in Mountain City are cars and the burning of fossil fuels. They release toxic chemicals into the air such as CO and CO2. This greenhouse gas is contributing to global warming and that is harming the environment in its own way. Some ways to combat this are taking public transportation with electric systems or electric cars. If we work together, we can stop pollution for good.

Similarly, Jason had a successful experience using metacognitive scaffolding to support the creation of arguments. Since he was assigned the stakeholder role of an asthma sufferer, an issue from which he suffered himself, Jason demonstrated an increased passion for looking up information that would substantiate his claims (i.e. considering what, who, and how), while representing the perspectives of his stakeholder. Jason defined the problem from this perspective (i.e. task 1) with information collected by VFTs and utilized facts in support of his claim that if ozone issues were addressed in the local city, and the number of asthma patients could be reduced (i.e. task 5). In addition, several other data sources (e.g. students’ essays, screen recordings, and notes) indicated that the group he led generated claims with strong evidence about the air-quality solution for asthma sufferers.

Negative Results.

Some students did not take advantage of the scaffolding. They indicated a desire to find answers as quickly as possible to finish their tasks, similarly to what they were used to doing during teacher-led instruction. As a result of their lack of engagement with the prompts, these students were unable to develop strong arguments to support their solutions to the given problem in the Connection Log. As an example, Sara did not consider the perspective of her stakeholder when defining her problem (i.e. task 1) and continuously asked: “what is our stakeholder?” Recordings showed her written problem statement as “Mountain City’s air quality is so serious,” which does not demonstrate in-depth thinking over the metacognitive scaffolding prompts (i.e. what, who, and how). Sara did not reflect on factors that can largely affect the quality of one’s argumentation, such as those prompted during tasks 1 and 4. Furthermore, her disregard for properly completing the previous tasks resulted in a claim that was not supported by relevant evidence (i.e. task 5).

Strategic Scaffolding.

The scaffolding program guided students across all tasks. That is, as learners organized findings, they were prompted to break previously found information into categories, specifying what is happening and how it affects the stakeholders (i.e. tasks 1 and 2). Then, students worked in groups to eliminate irrelevant information and develop claims (i.e. tasks 3 and 4). Afterwards, students were prompted to write their claims individually, compare them to those of their groupmates (i.e. task 4), and come to a group consensus on what claim to present. Finally, students working on linking evidence to claims were individually prompted to categorize and test the evidence supporting each claim, and then, as a group, to combine evidence and claims and form arguments (i.e. task 5).

Positive Results.

Evan’s argumentation skills showed improvement from pre- to posttest. He took advantage of the Connection Log, which he considered a helpful tool in developing arguments and proposing solutions for the problems assigned within PBL. Although he initially viewed science as boring, he demonstrated active engagement in the unit. Notably, Evan collected information, organized it, and, based on the evidence he found, came up with a justifiable and validated solution about which he was very confident (i.e. tasks 4 and 5).

Evan: I’ve learned that making a good solution totally depends on information. I think that if information is believable, my solution supported by this information can be powerful. I just followed the steps in the Connection Log, and each step told me what I should do to make a claim.

Negative Results.

Two students in group 4 had negative experiences using the scaffolding program; these were the only two students to score lower in both information literacy and argumentation skills posttests. Cathy, who was mostly familiar with teacher-led instruction, struggled to understand tasks at the beginning of the PBL unit. Her responses to interviews often included statements such as “I did not know what to do” or “I was frustrated.” Screen recordings indicated that she did not make much use of strategic scaffolding provided by the Connection Log and instead just went through the five tasks without writing any entries. If there was an error message due to no entry, she tended to react emotionally, saying “I hate science” and “so much to do.” According to Cathy, the strategic scaffolding made the procedure complicated and repetitive. This contributed to her development of poor-quality claims, not linked to any evidence. An example of her disinterest is demonstrated in her final claim, “want to eat pizza,” which was not at all relevant to the topic. Another student, Charles, was dissatisfied with the prompts in the group portion of the PBL tasks. According to him, he felt it was “a waste of time” to join the scribe to input final claims into the software, and he “didn’t understand why [he] should do that. Because he did not work alongside his peers and did not concentrate during the collaborative portion of the assignment, he was unable to take full advantage of the PBL experience, and instead, perceived that the strategic scaffolding “made it difficult for [him] to finish [his] work.”

It is worth noting that the students’ negative experiences reflected in these findings might not be closely related to strategic scaffolding per se, since these students had expressed a preference for teacher-centered instruction and stated that they did not like science.

Discussion

PBL students need both information literacy and argumentation skills to enable them to search for, evaluate, and use information effectively and consequently generate coherent arguments. This study aimed to fill a gap in the literature regarding (i) whether or not scaffolding impacts students’ information literacy and argumentation skills, and (ii) the positive and negative effects resulting from the different types of scaffolding. VFTs and the Connection Log were designed and developed to provide scaffolding that supported students in addressing their difficulties in terms of conceptual, metacognitive, and strategic aspects of the learning process.

Quantitative results demonstrated the strong relationship between information literacy and argumentation skills required for successfully engaging in PBL. All participants who attained improved scores in the information literacy pre- and posttests also performed better in the argumentation skills pre- and posttests. This supports the idea that information literacy is required to create evidence-based arguments for problem-solving in PBL (Cardetti & LeMay, 2019; Moselen & Wang, 2014). Results from this study were consistent with the literature, confirming that scaffolding within VFTs can significantly improve students’ information literacy by enhancing their ability to find, evaluate, and utilize information (Wiley et al., 2009). Scaffolding also had a positive effect on argumentation skills, enhancing students’ ability to organize their thoughts and develop strong arguments. Moreover, our study revealed that scaffolding functioned differently among students according to each student’s learning status and needs. That is, although every student received the same scaffolding, each learner was able to ask for scaffolding if needed; thus, experiencing scaffolding in their own way. These varied experiences support the idea that diverse students can use scaffolding in different ways on the basis of distinct goals and prior experiences (Belland et al., 2019). These findings shed light on the fact that students were able to perceive the affordance of scaffolding in that they realized what could be accomplished using this support and how to use it accordingly (Belland, 2011).

The analysis of the three scaffolding types vis-à-vis their effects on the learning process revealed positive results for students working on tasks within the VFTs and the Connection Log. The successful scaffolding interventions within the VFTs enhanced the majority of students’ information literacy, improving their ability to effectively use information for creating valid arguments for problem-solving. Students with the greatest improvement of information literacy (groups 1 and 2) took full advantage of conceptual, metacognitive, and strategic scaffolding, receiving support in the following tasks: (a) gathering information from the perspective of different stakeholders, (b) reflecting on their learning and understanding of content, and (c) validating and effectively utilizing gathered information. In short, because simply collecting and utilizing information to solve problems does not guarantee improved information literacy (Manny & Ellis, 2019), scaffolding from the VFTs invited students to expand their thinking and realize that determining the solution to bad air quality involves more than just collecting information on air-quality indicators (Wiley et al., 2009). Similarly, students who showed the most positive effects in their argumentation skills (groups 1 and 3) were those who fully made use of the support provided by metacognitive and strategic scaffolding as they completed tasks in the Connection Log. The two scaffolding types enhanced students’ ability to organize their thoughts and to develop strong arguments to solve air-quality issues, as confirmed by interview data (Looi & Lim, 2009). The most effective scaffolding types guided students in considering who, what, and how, and encouraged them to validate the gathered information to develop strong claims to solve problems regarding air-quality issues, linking these to evidence. Another noteworthy finding is that, because scaffolding had different effects on participants and supported them in accordance with their desire to take advantage of these resources, the quality of outcomes changed depending on how well each student completed their tasks (Noroozi et al., 2018).

On the other hand, results also revealed negative effects stemming from the use of computer-based scaffolding. However, the results of this study differed from the negative patterns identified in the literature (Rashid & Asghar, 2016; Rosen, Whaling, Carrier, Cheever, & Rokkum, 2013) since the students in this study did not lack technological skills or have negative attitudes towards technology. Instead, students’ reactions during the VFTs demonstrated a dislike towards the abundance of pop-up windows. Likewise, some students (group 4) regarded the hints, prompts, and feedback from the Connection Log as a hindrance to their learning. They thought that the structure provided by the software made learning more difficult, partly because of the repetitive aspect of the prompts, which reduced motivation to continue working (Simons & Klein, 2007), and partly because of learners’ personal preference for teacher-led instruction heavily influencing how they viewed and approached scaffolding during the unit (van de Pol, Volman, & Beishuizen, 2010). That is, scaffolding provided general strategic support (i.e. the systematic steps of PBL), as opposed to more specific answers to questions, which possibly hindered some students from successfully gathering information or making arguments. Another probable explanation for the negative findings lies in these students’ need to take a credit recovery course. The combination of retaking a class and their reported disinterest in learning science would corroborate their inappropriate use of the tools and lack of improvement in this study.

Apart from the few negative aspects found in the three types of scaffolding, and with the caveat that generalizing case study findings is limited, this study has important implications regarding the implementation of scaffolding within PBL and the future design of scaffolding interventions. When implementing scaffolding, teachers need to make learners aware of the benefits of this type of support while also reassuring them that they can ask their teacher for help any time they deem necessary, which can lead to an enhanced learning experience for different students. In addition, individualized scaffolding should be provided to meet each student’s needs. In this respect, there are currently ongoing artificial intelligence research projects designed to meet the growing demands of individualized learning support systems in science education. These aim to give students immediate feedback and support catered to their individual understanding of content and various ability levels (Belland et al., 2017, 2019; Poitras, Butcher, Orr, Hudson, & Larson, 2019).

Conclusion

This study investigated the positive and negative effects of three types of computer-based scaffolding (i.e. conceptual, metacognitive, and strategic) to support high-school credit recovery students’ information literacy and argumentation skills when engaged in a problem-based learning unit on air quality. Results revealed that the three scaffolding types facilitated credit recovery students’ inquiry-based learning process in different ways, enabling them to realize the affordance of scaffolding in supporting the creation of evidence-based claims. Conversely, this study’s negative findings can enlighten researchers of challenges involved in utilizing scaffolding in PBL lessons. This, in turn, may help teachers and instructional designers to create tasks that effectively stimulate information literacy and argumentation skills for a successful inquiry-based learning experience.