Introduction

Raising the level of student science achievement often falls to the science teachers within the classroom and parents outside of the classroom. Administrator, politicians and the public continuously ask science teachers and teachers in general, to deliver an ever-growing curriculum with less time and resources. This increased demand often times can result in lower reported teacher job satisfaction (Johnson 2003; Pas et al. 2012; Xaba 2006). One possible solution to this growing pressure on teachers is to increase the use of online laboratory instruction integrated in high school science courses (Barak 2002; Guzey 2009).

The National Center for Educational Statistics reported in 2001 that 99 % of high schools within the United States had some form of Internet access. Internet access has only continued to increase over time. The report went on to say that of the 99 % of schools who had Internet access, 96 % of those schools had connections allowing for online simulation use (Cattagni 2001). With the near ubiquity of the Internet in schools, along with the numerous science-themed websites available to teachers, there have been proposals that some modes of online laboratory instruction would be beneficial in raising student achievement (Card 2006; Gagne 2005; Guest 2001; Shadish and Sweeney 1991; Shieh 2011; von Glasersfeld 1996).

For the purposes of this study, the authors define traditional instruction as lecture, drill-and-practice or educator-centered learning and teacher-led instruction. An operational definition of computer/technology-aided instruction, or non-traditional instruction, is one in which there are learner-centered modes of education in which the learner makes use of a computer, software or Internet websites to interact with and learn science-based content. The authors define online laboratory instruction as simulation-based instruction, which seeks to replace, or in the very least augment, existing, hands-on laboratories. These simulations also act to enrich and connect previous instruction.

Purpose, Research Questions and Hypothesis

The purpose of this study is to investigate the effects of online laboratory simulations as a mode of instruction versus traditional laboratory-based chemistry instruction in a high school chemistry classroom. The study also sought to uncover underlying factors that affect student outcomes while learning chemistry content using online laboratory simulations. The authors of this study suggest that computer simulations (Curriculum Pathways, CP) seem to produce conditions in which student understanding of conceptual information increases more than when students receive traditional instruction alone. The research questions, addressed in this study, are as follows:

  1. 1.

    Do high school student chemistry outcomes vary across the factors of teacher experience and socio-economic status (SES) when using online laboratory simulations?

  2. 2.

    What is the effect of an online laboratory simulation intervention over time?

  3. 3.

    What is the likelihood that high school students using an online interactive laboratory are successful on a student assessment by treatment group?

  4. 4.

    Do high school students differ in their outcomes and attitudes regarding chemistry when considering teacher experience and treatment group?

Consideration of the research questions and the literature supports the following hypotheses: When the use of online laboratory simulations and traditional instruction are controlled, the online laboratory group will demonstrate a statistically significant increase over traditional instruction in their understanding of chemistry content associated with the technology-based instruction. A secondary hypothesis is that the factors of SES, Student Attitude and Teacher Experience will have differing impacts on student outcomes when compared across intervention groupings.

Literature Review

Effects of Multimedia and Interactive Web Sites on Student Outcomes

Modern software, such as interactive, multimedia websites, allows students to receive and process more information from electronic sources than they do through lecture and written forms (Connolly 2006; Malik 2008; Reeves 1992). Literature has also shown that students who engage in the use of multimedia websites during instruction show larger gains in content knowledge. In addition, these students seem to report higher positive attitudes regarding science and show increased attentiveness, interest and engagement in the educational process (Gilbert et al. 2003; Hannafin et al. 2009; Lamb and Annetta 2009; Lamb et al. 2011). As model-based teaching creates and emphasis on 21st century skills such as virtual interactions and manipulation, there are more and more attempts to reach digital natives with websites designed to aid online simulation modes of instruction. This has resulted in laboratory simulations playing a larger role in the science (chemistry) classroom (Najafi et al. 2007; Windschitl 2000).

A meta-analysis conducted by Bayraktar (2001) revealed that 41 of 42 studies between 1971 and 1999 demonstrated the positive effects of the use of computer simulations in science education on student achievement. A second meta-analysis of 52 studies conducted in 2007 by Liao also supported the previous meta-analysis conducted by Bayrektar and concluded that computer-based laboratory instruction in science education has a positive effect on student achievement.

Several authors including Flinn and Gravatt (1995) explain that, of the multiple types of computer instruction for learning science content, the most effective style is simulation and interactive models such as Serious Educational Games (Holzinger et al. 2009; Bell and Trundel 2008; Moreno and Mayer 2007). Other studies conclude that the second most effective style of teaching was tutorial. Lastly, the least effective form of computer instruction was drill-and-practice. In fact, there are studies suggesting that the use drill-and-practice (non-interactive activities) in conjunction with computer instruction can negatively affect student outcomes on content examinations (Howard 2009; Liao 1998).

Studies suggest that the use of technology in the classroom with a student-centered model (interactive and dynamic) can effect two very important changes in the classroom. One is a transition from traditional, teacher-centered instruction to other forms of instruction that are learner-centered. The second important change is the increased engagement and interest in the sciences. Examples of learner-centered areas of instruction are problem-based learning using computers laboratory simulations, inquiry-based laboratory and student-directed study (Annetta 2007; Lamb and Annetta 2010). These pedagogical approaches can lead to significant increase in the student’s perceived and actual success (Shuell and Farber 2001). The student initiates this perceived and actual success through computer feedback tied to the “reward” system. It is the stimulation of this reward system, which provides the impetus for student engagement (Howard-Jones and Demetriou 2009). Correctly answering the question results in positive outputs from the computer software, while incorrect answers results in negative outputs from the computer software (Anderson et al. 2003; Frost 2007; Van DeGrift et al. 2002).

Advantages of the online laboratory over traditional laboratory instruction are that they (online laboratories) allow for more tightly controlled experimental variables. This level of variable control gives students the opportunity to see and analyze laboratory outcomes, which are unclouded by distracters interfering with meaningful learning. Online laboratories also allow students to run experiments multiple times in a compressed time-period, giving students more opportunity to experiment, explore, adjust variables and see connections between concepts in chemistry. These opportunities are many times unavailable within the traditional laboratory experiences due to time and resource constraints (Snir et al. 1993). Lastly, within the online laboratory, students have the opportunity to work with laboratory equipment in a virtual environment that may not be available to them due to cost, complexity of use, lack of training or due to resource constraint (Rozewski et al. 2012; Shen and Hong 1999).

Online Simulation and the Constructivist Approach

Constructivism is a view in which a student builds his or her knowledge of phenomena using interpersonal and individual experiences through reflection (Fosnot 2004; Gil-Pérez and Gil 2002; Jenkins 2000; Phillips 2000; von Glasersfeld 1984; Hofstein 2004; Windschitl and Andres 1998; Yager 2000). The group approach fostered by online computer simulations and laboratories help to build the social context of learning. The social context of leaning is vital to the constructivist framework in education. In this model of learning, the learner’s mind is not an isolated storage facility for facts. Instead, the learner accomplishes “meaning making” within the contexts of peers, relationships and the environment (Windschitl and Andres 1998). This “meaning making” typically will manifest itself, for the constructivist, as the need for learners to work within group settings. In these settings, individuals will be able to co-constructing knowledge. They need not always reach consensus as long as they are able to help each other construct, or reconstruct as necessary, the new knowledge (Pepin 1998). The use of computer simulation supported by the constructivist approach to learning offers a more positive response to learning (content) than through more traditional approaches to learning (Windschitl and Andres 1998).

Methodology

Sample

The subjects selected for this study were 351 students in eleventh and twelfth grade ranging in ages from 16 to 18 years old. All subjects had completed high school Biology and Algebra I. There are 201 students assigned to the experimental group and 150 students assigned to the control group. The study authors further divide the subjects by socio-economic status (SES) with 93 students within the control group reporting a low SES and 57 reporting a high SES. SES level was established using a self-reporting of free and reduced lunch status. The experimental group shows a differing division with 84 subjects reporting a low SES and 117 reporting a high SES. One should note that a majority of the control group identify as low SES, while a majority of the experimental group identify as high SES; the authors account for this during the statistical analysis though mean weighting. Gender breakdowns for the groups are as follows: 59 females and 91 males within the control grouping, and 79 females and 122 males are placed in the experimental grouping. The sample is representative of the school population and illustrated in Table 1.

Table 1 Demographic make up of the population and sample
Full size table

Design

The study design is a non-randomized, pretest–posttest, control, intact, group design. This design has a primary benefit of more closely aligning with “normal” classroom and schools conditions. While the study participants are not randomly selected from the target population, the intact class is randomly designated for either intervention or control groupings randomly. This design also tends to minimize disruption of student learning and teacher planning. The researchers assigned the participants to one of two treatment conditions. The two treatment conditions are traditional instruction on three areas of chemistry outlined in the “Procedure” section below (control). The authors define traditional instruction as methodologies such as lecture, drill-and-practice and educator-centered teaching and learning. The control group was in place to account for the manipulated variable of technology use in the classroom. The second condition is that of the experimental or intervention group. The teachers gave each of the students within the experimental group access to the online laboratory simulation in place of a block of instruction. Table 2 outlines the research design.

Table 2 Study design
Full size table

The pretest–intermediate test–posttest, control, intact, group, study design allows for the examination of the treatment effects from differing perspectives. The use of this design does have one serious threat to internal validity, that of selection bias. The authors mitigated the design threats through matching of characteristic of computer use in the classroom. The use of intact group designs allows for the reduction in the interaction of group selection and treatment due to the difference in setting. A second benefit of this design is the reduction in the reactive effects of experimental procedures (Table 2).

Intervention Description, Curriculum Pathways

Curriculum Pathways (CP) is a web-based online laboratory and instructional tool that is based on the constructivist approach to learning. The creator of Curriculum Pathways intends it to assist classroom teachers by providing students with content instruction via web quest and online simulations of chemistry phenomena at the macroscopic- and nanometer level. The scalar and phenomenological approaches act as an attempt to challenge student conceptions via creation of disequilibrium. Table 3 illustrates gain score mean and standard deviation by gender, by experience, by group. CP also provides for online laboratories and group interactions via focus questions, postings and group discussion prompts. One of the major components of the CP simulations is the use of non-traditional chemistry models to aid in instruction.

Table 3 Gain score mean and standard deviation by gender, by experience, by group
Full size table

CP sets the conditions for the exploration of laboratory content while interacting with macroscopic-level and nanometer-level simulations of chemistry phenomena. By challenging student conceptions at multiple scale levels, CP causes individual students to experience disequilibrium; because of each student’s disequilibrium, the whole group experiences disequilibrium. CP then allows students to discuss and develop a social consensus to reestablish equilibrium within the individual student and group. The vague and open-ended focus question prompts students to work through this reestablishment of equilibrium. This process is equivalent to the use of problem-based learning grounded in the constructivist approach to teaching and learning (Annetta 2007).

Three simulations (modules), two of which contain a computer simulations of selected laboratory topics, and one module contains an external web quest tied to laboratory simulations, were presented to each of the experimental group participants as part of the instructional unit.

The first module (The Mole) presented users with a list of sixteen web-based resources containing simulations using the Mole as a unit of conversion and measure in chemistry. Students completed the web quest and associated laboratories by interacting with each site and using the laboratory simulations found with each of the websites. Upon completion of each of these simulations and laboratories, the students answering online feedback-based questions designed to provide feedback for learning.

The second module (Chemical Reactions) presents users with a laboratory simulation of a common laboratory conducted in high school chemistry. Throughout the laboratory, CP uses guiding questions to set the conditions for continuous feedback and learning. The laboratory provides users with two dropdown menus containing soluble salts. The user then selected a combination of salts to react in solution. The simulations represent the aqueous salts using pictorial representations of the individual ions and compounds at the nanometer-level. When the user has correctly completes the reaction, a video of the reaction occurs which shows if a precipitate was formed at the macroscopic level. The student records this information in answer spaces provided within the laboratory simulation. When all guided questions and trials were completed, the simulation presents the students with their product the solubility rules for double replacement reactions.

The third module (Stoichiometry) presents users with a laboratory simulation designed to review stoichiometric relationships using limiting reagents. The user selects from eleven reactions represented via chemical equations and molar ratios. While interacting with the laboratory simulation, the users explore how reactant quantities affect reaction outcomes in a balanced and unbalanced chemical equation. Students completed each reaction as explained in the instructions for identifying the limiting reagent. Based upon their identification of the limiting reagent, students then write out and “build” the reaction using a stockpile of atoms while observing the molar relationships and oxidation numbers. All three of these interventions allow students to view multiple representations of chemistry.

Measures

The content assessments for each of the modules consisted of five multiple-choice questions and five open-ended response questions for a total of thirty content-based questions. The subjects also answered an additional self-reporting attitudinal survey consisting of 26 questions and 5 demographic questions. Researches gave each of the students using the simulation a pretest, intermediate test and a posttest. One potential threat using this approach is the possibility of potential carry-over effects. To mitigate the carry-over effect, the researchers scrambled the test questions and multiple-choice answers between iterations of the test before giving it to the students. The test for each module was identical with the exception of scrambling of the questions and answers. The study authors estimated the internal reliability of the measure using Cronbach’s alpha. Results illustrate an estimation of a Cronbach’s alpha of 0.39; this low estimate was likely due to random guessing by the subjects of the study and the diversity of the areas addressed by the test. The internal consistency of the intermediary test and posttest was α = 0.86 and α = 0.87, respectively. The increase in internal constancy of the test is likely due to more consistent answering of the questions as student understanding of the content increased. The increased internal consistencies decrease error variance and make a Type II error less likely. Thus, significant effects are less likely due to chance.

The creators of the measure established content validity through analysis of alignment to the National Science Standards for Physical Science (NS.9-12.2). Two chemistry instructors and a professor of science education reviewed the tests for clarity of the questions, appropriateness in regard to the content reviewed in the simulations and taught during instruction and alignment to the National Standards. Appendix A contains the pretest, intermediary test and posttest items used to assess the students’ content knowledge. In addition, to the content tests, teachers gave students journal prompts to answer. The journal prompts promoted student reflection of the factors that affect attitude and outcome when using the online simulation. Appendix A contains the measure items and journal prompt questions.

Procedure

Students participated in their assigned class sections; there were five sections with approximately thirty students each. The researchers assigned each class at random to the control (traditional instruction) condition or the experimental condition (technology-aided instruction).

The study took place over a 9-week period during the school year. The school year started in January and ended in June. Each semester consisted of two 9-week quarters, with each class on a 4 × 4-block schedule with 90-min per class. Teachers had given the students instruction in the use of the simulations prior to the start of the study.

At the start of each laboratory (Module 1, The Mole; Module 2, Chemical Equations and Module 3, Stoichiometry), students in the control and experimental groups were given a multiple-choice and open-response pretest to assess prior knowledge in regard to the chemistry content. The classroom teacher presented each group with either the traditional laboratory instruction (control) or the online laboratory simulation (experimental). Content for the first unit of institution focused on quantitative chemistry without the use of online simulations. The teacher-led instruction consisted of 30 min of instruction followed by 60 min of hands-on activities. The classroom teacher then gave the experimental group access to the online simulations for the 90-min period. Teacher-led instruction was limited to a maximum of 30 min. Upon completion of the 30 min of instruction, students took the intermediate test to assess the progress of their learning. The participants then took part in the laboratory experience. Following the laboratory, teachers gave each class a posttest and the journal prompts. The remaining two modules followed a similar instructional and assessment schedule over the remaining 6 weeks.

Data Analysis and Assumptions

The study authors conducted analysis of affective factors, demographic factors and content outcomes using multiple types of analysis to gain perspective on the aspects affecting student outcomes in chemistry. The four methods used to conduct the analysis are a Three-Factor Analysis of Variance, Times Series Repeated Measures Analysis of Variance, Logistic Regression and Multiple Analysis of Variance. The authors used a general qualitative approach to analyze the journal prompt responses (Corbin and Strauss 1990).

The study authors quantified the Chemistry Data from each of the measures the content test scores, content questions were either correct or incorrect with no partial credit awarded. Correct scores on the assessments were give one point and incorrect scores given zero points, blank items are incorrect and received zero points. In an effort to calculate overall content learning, the authors used gain scores between each of the tests.

The students answered a self-reporting survey coupled with the journal prompts as a means to collect socio-economic and gender. For the purposes of this study, teacher experience dichotomization occurred in the follow manner, less than 5 years of teaching experience is novice (0) and greater than 15 years is experienced (1).

Student completed questions (prompts) within their journals at the end of each module as a means to collect attitudinal data. The teachers then collected the journals at the end of the study, and the authors reviewed them for common trends. Concept codes for key phrases describing chemistry Internet access, comfort/easy of computer use, information availability and novelty of computer use in class were the target of the qualitative review. Overall, attitudinal scores derived from a simple comparison of positive statements versus negative statements. If the number of positive statements was greater than the number of negative statements, the reviewers considered the subject to have a positive attitude. The reviewers then grouped the results from the attitudinal review into common concepts. From these concepts, the reviewer suggested trends. Triangulation of the results occurred from the commonality found in each method result. During the analysis, the authors accounted for differences in SES sample size through segment weighting of the means. Segmentation weighting equalizes the differences between the two sample sizes a ratio of the mean, thus more closely approximating each group’s true impact; in addition, the author cross-validated the results of the ANOVA using ANCOVA to account for variation in the SES of the subjects.

The initial analysis conducted by the authors was at three-factor Analysis of Variance (ANOVA) addressing research question one. Research question one explored the variance of the factors of teacher experience and socio-economic status which using an online laboratory simulation. The outcome variable was the chemistry student content gain score across the factors of teacher experience (novice or experienced), SES (high or low) and grouping (control or treatment). The primary assumptions of ANOVA are independence of observations, normality and homogeneity of variance or equality of variance across the sample.

The second data analysis method used was the time series, repeated measures analysis of variance (TANOVA). The TANOVA used during the analysis addressed research question two. Research question two explores the effects of online simulation interventions over time. Outcome variables for this measure were individual chemistry content pretest, intermediate test and posttest assessment scores as a function of multiple time points.

Logistic regression results answered research question three. Research question three seeks to address the likelihood that high school students using an online laboratory simulation are successful while using an online laboratory simulation. The dependent variable for this analysis is that of success on a chemistry content assessment (pass or fail) as a function of the treatment group. Passing for the purposed of this study is a student score of six out ten on each content assessment section.

To answer the fourth research question, the authors employed Multivariate Analysis of Variance (MANOVA). Research question four seeks to explain student outcomes attitudes regarding chemistry when considering teacher experience and treatment group. Outcomes variables for the MANOVA are gain scores and dichotomous attitudinal scores (positive or negative).

Results

Quantitative Results

Results for the analysis of factors related to research question one show that the three-factor ANOVA has no statistical difference between student gain scores on chemistry content across the main effect of teaching experience F (1, 350) = 0.40, p = 0.533, respectively, α = 0.05. However, there is a statistically significant difference across the main effect of grouping control versus intervention F (1,350) = 3.94, p = 0.042, α = 0.05. The Pη2 for this analysis is 0.110. The 95 % confidence interval for this difference shows that the treatment group outperformed the control group by a score 3.03–3.67. There is no evidence of statistically significant interaction effects across factors of teacher experience and grouping, F (1,350) = 0.03, p = 0.960, α = 0.05 scores by experience and grouping. Table 4 shows the analysis of variances outcomes for each of the discussed variables.

Table 4 Analysis of variance for chemistry performance (gain score) by gender, experience and grouping
Full size table

Analysis of factors related to research question two show that the TANOVA levels of the means and standard deviation for pretests (M = 1.22, SD = 1.28), intermediate test (M = 3.41, SD 1.70) and posttest (M = 4.38, SD = 1.91) are statistically significant across the factor of time. The results indicates under the conditions of time at α = 0.05. Specifically, Wilk’s Λ = 0.07, F (2,350) = 334.59, p < 0.001, mη2 = .93. Given a statistically significant Wilk’s Λ, it is possible to omit the univariate table, measuring the test of sphericity and within-subject effects. As can be seen, the contrast between level 1, 2 and 3 is statically significant. Figure 1 shows the profile plot for each measure over time. Table 5 summarizes the result of the TANOVA.

Fig. 1
figure 1

Estimated marginal means over times

Full size image
Table 5 TANOVA on gain score across time
Full size table

In an attempt to answer research question three, the authors used binary logistic regression. Predictions for passing the chemistry assessment are statistically significant for grouping X 2 (1) = 63.539, p < 0.00, R 2 = 0.46. For every unit of increase, the odds for passing content assessments increase by a factor of 1.38 (about one and a half times) for every one unit (60 min) of online laboratory instruction when controlling for all other factors. The effect of these results is within the 95 % confidence interval range of (0.825–2.146). Table 6 summarizes the results of the logistic regression.

Table 6 Logistic regression analysis of student success on chemistry content tests as a function of grouping and gender: passing versus failing
Full size table

Answers to research question four employed a planned comparison multivariate analysis of variance. The results show that the unique contribution of group is statistically significant on the set of two dependent variables chemistry assessment outcomes and attitude. Resulting linear discriminate functions (LDFs) and centroids displayed in Table 6 illustrate the relationships. The LDFs provide information regarding the maximally separated linear combinations of dependant variables separated by treatment groups with Eigenvalues for function one accounting for 65.9 % of the variances across the dependant variables for the four groups of students. Eigenvalues for function two account 34.1 % of the dependant variables for the four groups within the study (Table 7).

Table 7 Statistics for LDF 1 and LDF 2
Full size table

Results for the first and second linear discriminate function are statistically significant Λ = 0.47, X 2 (198) = 597.72, p < 0.001 and Λ = 0.47, X 2 (153) = 187.72, p < 0.001. The dependant variables Y1 Student Assessment Outcomes correlates with LDF 1 and Y2 Student Attitude correlates with LDF 2. Figure 2 shows the centroid location across LDFs. The significant results are indicated by (a) Wilk’s Λ = 0.47, F (6, 3,347) = 697.62, p < 0.001 and (b) on each dependent variable separately as indicated by univariate F test for chemistry assessment outcomes F (3,347) = 69.76, p = 0.04 and attitudinal outcomes regarding chemistry F (3,347) = 83.88, p = 0.02. The results presented in this study can be readily interpreted as the Box’s M test is not statically significant X 2 (9) = 5.38, p = 0.80, indicating that the multivariate assumptions of equal variance–covariance matrices are met assuring the validity of the results.

Fig. 2
figure 2

Group centroids for four groups separated by two linear discriminate functions

Full size image

Discussion

Quantitative Discussion

The primary purpose of this study was to compare traditional instruction and online laboratory simulations. Secondarily, the study also examines several demographic factors and altitudinal factors, which may affect outcomes. The authors explore the question; do online laboratory simulation groups (treatment group) demonstrate significant increases in learning over traditional instructional approaches measured using chemistry content assessments. A secondary hypothesis is that SES, student attitudes regarding chemistry and teaching experiences mediate the effects of online laboratory simulations. The results of the study analysis suggest mixed outcomes with respect to the mediating effects. The lack of a significant difference in the pretest scores for all simulations shows no one group of students had more prior knowledge than the other group of students. Thus, all intact units worked from the same level of prior knowledge as the start of each topic.

Research question 1, do high school student chemistry assessment outcomes vary across the factors of teacher experience and socio-economic status when using an online laboratory simulation, made use of an Analysis of Variance (ANOVA). Results of the ANOVA suggest that there are no differences in student gain scores across the factors of experience and grouping. However, effects of grouping (treatment vs. control) show statistical significance indicating that the online laboratory simulations play a role in student outcomes on chemistry content assessments. Further to this point, there is an implication that other factors, which normally affect student learning, moderate during the use of online laboratory simulations. Students who use the online laboratory score in the range of 31 and 37 % higher than students who only have access to traditional forms of instruction. An effect size of 0.11 (relatively moderate) for the comparison of the online simulation gains versus traditional instruction indicates the influence of the intervention on student learning.

These outcomes for the ANOVA further supported by the results of the logistic regression reinforce the role of online simulations in student learning. Research question 3, what is the likelihood that high school students using an online interactive laboratory are successful on a student assessment by treatment group, was answered using this statistical method (logistic regression). Outcome of the logistic regression indicate that the treatment group was almost two times more likely to show success on chemistry content evaluation. Implications for the results of the logistic regression show that a minimum of 60 min of interaction with the online simulation raises student posttest scores. This is significant as it quantifies necessary instructional requirements for students to pass assessments for each of the topics.

Research question 2, what is the effect of an online laboratory simulation intervention over time, indicated the use of time series analysis of variance (TANOVA). Result of the time series analysis suggests sustained and continuous growth of student learning over the period of intervention. This indicates that the chemistry content assessment performance under the treatment conditions continues to increase with an effect size of 0.93 (very large). Thus, these results successfully address research question three and provides for the effect of the laboratory simulation over time.

Research question 4, do high school students differ in their outcomes and attitudes regarding chemistry when considering teacher experience and treatment group, was answered using Multivariate Analysis of Variance (MANOVA). Results of the MANOVA suggest that there is significant chemistry content assessment outcome differences between the control groups and the intervention group regardless of teacher experience. The reader should note that in the study, there was a slightly negative outcome on content assessments (Function 1) when using traditional teaching methods. This result supports the results of previous studies suggesting negative outcomes for traditional methods of instruction. The authors attribute the difference between traditional instruction and online instruction using simulations to student disengagement during the learning process. Multiple analyses types support the significant outcomes and maintenance of the effects over time for the intervention groups. Given the isolation of the intervention variable (online laboratory simulation use), there is considerable evidence that this difference was in fact a result of the intervention. The data in the study indicate the need for further cognitive studies to assess the underlying cognition process of the students. These future studies may incorporate the use of cognitive diagnostics to ascertain effected student cognitive attributes.

Qualitative data results support ascertains that the novelty and interactive nature of the intervention helped to set the conditions for change in student outcome. More significant is the difference between intervention and control groups on the attitudes of students. These attitudinal differences seemed to occur regardless of the experience level of the teacher in the classroom. In fact, analysis of the MANOVA plots illustrates little difference in the outcome based upon experience levels within the intervention group. There is, however, a significant attitudinal difference between the following groups: Novice Control (NC) and Experienced Control (EC). These differences in the attitudinal outcomes tie to individual teacher experience when using traditional teaching methods. While the lack of differences seen in the intervention groups seems to imply that the use of online simulation helps to mitigate experiential differences between teachers. In this particular, study the Novice Experimental (NE) and the Experienced Experimental (EE) place in similar location on the centroid figure. Overall, this suggests that there are unique contributions due to attitude and content functions as both experimental groups show larger differences along the functions of both the attitude and the assessment. This indicates that the laboratory simulations show positive and larger impacts on both groupings simultaneously while the traditional instruction only show negative attitude shifts when reviewed in the light of teacher differences. This provides supporting evidence for contentions that online simulations create conditions for the development of positive science attitudes and outcomes.

Each of the analyses addresses specific aspects of student chemistry content outcomes and attitudes. The quantitative analyses reveal that there are significant outcome impacts resulting from the use of online laboratory simulations within the science classroom. These analyses also seem to indicate that common underlying demographic factors are not significant when considered in the context of online laboratory simulations.

There are several reasons why students engaging in the use of online laboratories may lead to increases. The online laboratories may provide for more opportunity to explore and more iteration allowing learners to link concepts in the online simulation. This repetitive approach to laboratories provides more available information in multiple forms such graphs, text, pictures and interaction than is otherwise possible during traditional laboratories. Furthermore, the use of an online virtual manipulative allows for more rapid cycling of support and refutation of student understanding within chemistry. This self-explanation can lead to greater understanding of content. Thus, using self-referencing through externalized validation via online simulations provides for more opportunities to engage in consensus checking with the computer. The self-explanation acts as a meta-cognitive strategy for student learning and increases student understanding. Results support a previous study by Aleven and Koedinger (2002).

The underpinnings of the apparent increase in the experimental group result from the vague open-ended questions that act as the catalyst for student discussions and allowed students to develop a social consensus to reestablish equilibrium within the individual student and group. This opportunity for construction and establishment of individual and group equilibrium did not necessarily present itself while traditional instruction was occurring due to proximity of the authority as the center-point of the classroom.

The constructivist approach differs from the traditional approach in several ways; it is difficult to conclusively to tie the conceptual change, and the increased student gains in the technology-aided instruction to particular aspects of one condition versus the other. Isolation of the particular constructivist elements of the online simulation may cause the loss of resolution of the whole interaction associated with the technology.

Conclusion

This study provided evidence that the use of online simulation acts as a means for increasing student’s chemistry content understanding and attitude toward science. Evidence in this study is consistent with the view that providing students with interactive, simulations using open-ended questions may help to provide the catalyst for conceptual change through constructivist approaches and promotes increased understanding of chemistry content. The study also suggests that the demographic factor differences across such factors as SES, attitude and teacher experience moderate with online laboratory simulations. In addition, journal entries suggest that informational load, computer familiarity and novelty of the computer use are factors, which can affect student outcomes on assessments.

Implication for Practice and Policy

Teacher factors such as the amount of time student’s work with the simulation are very important when considering the implementation of online laboratory simulations. School design and technology resource allocation should change to provide support to those teachers who use technology. While Internet penetration is nearly ubiquitous, school officials should redesign school policy and classrooms to accommodate technology use for individual and groups of students. A low student to computer ratio would greatly aid in module completion and allow the students to process information that otherwise may be rushed due to peer influence. Students could use and manipulate simulations multiple times and work through items that might be missed while working in groups.

School administration also needs to recognize the significant role that technology plays in the modern classroom, in particular, technology’s impact on student assessment outcomes. Administrations can more effectively design hiring criteria to better assess the level of technology use and understanding by the novice teachers, experienced teachers and potential hires. Further implications suggest that school leadership also needs to promote and produce professional developments that help to incorporate and integrate technology in the science classroom. The consideration of teacher technology experiences during the hiring of teachers can lead the school to effective utilization of technology in the classroom and ultimately result in better student outcomes on content assessments.

Implication for Technology-Aided Instruction

The study supports the assertions in the literature that the use of online simulations does help to increase student understanding of chemistry concepts. Wu and Shah (2004) suggested that the use of the computers and virtual models at the macroscopic and nanometer levels allows students to visualize chemistry by providing multiple representations and descriptions (Wu and Shah 2004).

By successfully visualizing chemistry through multiple representations, students are able to link their constructs to that of the content. This particular online simulation accomplished this through the representation of the interactive nature of chemistry through laboratory simulations. The laboratory simulations also allowed students to move between a traditional 2D representation of chemistry and 3D representations of chemical equations while using chemical modeling. The three simulations reduced cognitive load by making information more explicit and integrated within the students classroom work.

Computer simulation aids in the promotion of understanding of concepts through manipulations on the computer not otherwise available to students when taught using traditional pedagogy. This study further supports assertion that the development of a learner-centered education system with technology used as an adjunct significantly increases the achievement of students in the classroom. The study also supports the assertion put forth in the technological approach to which states that the use of computer models increases student achievement and understanding of conceptually difficult ideas.

This study shows that there is potential in the usage of online simulations to help students develop their understanding of chemistry. Some areas of further study are the use of multiple models and their effects on learners understanding of multiple chemistry topics.