Introduction

There are few areas of universal agreement among educators and the general public; however, most would agree with the idea that creating literate and successful students is a fundamental goal of education. This goal of creating literate students begins early. An extensive body of research has documented the significance of early literacy instruction and its effects on later academic success [e.g., Adams 1990; National Early Literacy Panel (NELP) 2008; National Institute of Child Health & Human Development (NICHHD) 2000; Snow et al. 1998; Sulzby and Teale 1991].

Because of the importance of developing early literacy skills, researchers have focused their efforts on identifying variables that support and facilitate early literacy success (Adams, 1990; Clay 1991; NELP 2008; Reutzel 2015; Snow et al. 1998). Overall, the strategies and environments utilized in early literacy classrooms are diverse; however, one tool that is used in most early literacy classrooms is technology. Recent research indicates that 98% of elementary school classrooms have computers in the classroom, with 75% of the teachers reporting that they use the technology regularly (National Center for Educational Statistics 2010).

Research on technology and literacy

“Technology” is a broad and somewhat vague term in education. Educational technology can refer to anything from software to hardware to a process to a product. Likewise, research on educational technology is wide-ranging and focuses on various applications, populations, and purposes. Because of the diverse focus of the research, it is often difficult to generalize findings and draw definitive conclusions about the role and effectiveness of technology.

Integrated learning systems (ILS)

The specific technological focus of this study is integrated learning systems (ILS). ILS are adaptive sequence systems that adjust instruction based on individual differences in students’ learning (Lee and Park 2007). These systems are fully integrated with the curriculum and are based on the concept of mastery learning. If a student masters a skill, the student progresses to the next skill. If the student fails to master a skill, the computer adapts and presents remedial information, reassessing until the student achieves mastery of the skill.

Research

Several researchers have noted the lack of high-quality research on the effectiveness of ILS on literacy achievement (Burnett and Daniels 2015; Cassady and Smith 2004; Paterson et al. 2003; Tracey and Young 2007). Many of the studies on ILS and literacy skills have produced somewhat mixed results, and it is difficult to draw definitive conclusions. The available research suggests that ILS generally have a positive effect on early literacy skills. For example, Bauserman et al. (2005) investigated the efficacy of PLATO’s Beginning Reading for the Real World on kindergarteners’ emergent reading skills. Their study found large effect sizes for phonological awareness and concepts about print (Bauserman et al. 2005). Both Tracey and Young (2007) and Cassady and Smith (2004, 2005) investigated the effectiveness of another popular ILS, the Waterford Early Reading Program. The results from these three studies indicated that the Waterford Early Reading Program had a statistically significant impact on young students’ early literacy skills, particularly their phonological awareness skills. Conversely, Paterson et al. (2003) studied the same ILS and found no benefits; instead, the researchers found that literacy facilitation by the teacher and time were more important to early literacy success.

Istation®

The ILS that is the focus of the current study is Istation®. Istation® is a privately held, Texas-based publisher with a portfolio of products that mainly focus on developing reading skills. The author has no ties or relationship with Istation®, related companies, or any of the researchers associated with Istation®.

An accurate count of Istation® users is unavailable; however, recent data suggest that over 4 million students in the United States use Istation® (PRWeb 2014). Three southern states have recently implemented and funded the use of Istation®, state-wide. Despite the high number of users, the statewide implementations, and the associated costs of Istation® Reading, there are no published reports on Istation®.

The content of Istation® Early Reading is organized around five domains of reading: phonemic awareness, alphabetic knowledge, vocabulary, comprehension and fluency (Mathes et al. 2012). These domains are based on the five pillars of reading presented in the National Reading Panel’s (2000) The Report of the National Reading Panel: Teaching Children to Read (NICCHD 2000).

Current investigation

Given the relevance of this topic to the educational system and the lack of research, the purpose of this present study was to investigate the effect of technology and teacher literacy support on the early literacy learning of young readers from an emergent literacy perspective. Specifically, this study investigated whether regular use of the Istation®, an integrated learning system, promoted the early literacy achievement of children in kindergarten classrooms. Another purpose of this study was to investigate whether Istation® is an adequate substitute for the more knowledgeable other (MKO) in the classroom. In other words, did this particular application of technology scaffold students’ learning as effectively as a classroom teacher and serve as a MKO? Generally, a MKO refers to a person who has a higher level of understanding and knowledge about a particular topic or concept (Vygotsky 1978).

The research questions considered for the current investigation were:

  1. (1)

    What effects does the Istation® reading program have on the literacy learning of kindergarten students? Is this learning significantly different than that of comparable children in classrooms without the Istation® program?

  2. (2)

    What effects does the level of literacy support provided by teachers in the classroom have on the literacy learning of the kindergarten students?

  3. (3)

    Is there an interaction between use of Istation® technology and the level of teacher literacy support in the classroom?

Theoretical framework

This study was framed using the ideas of Labbo and Reinking (1999), who suggest that studying literacy instruction and technology is a “process of negotiating multiple realities…because new technologies intersect with a broad range of issues and practices in literacy instruction” (p. 488). While the full range of issues and practices in literacy instruction are beyond the scope of this study, the data was interpreted in light of the complex relationship between technology and literacy, and multiple realities were considered using a mixed-methods approach.

There are two main ways to frame the relationship between literacy learning and technology. The differences between the two perspectives are subtle, yet important. One perspective is that children learn from a computer. Labbo and Reinking (1999) suggest that learning from a computer “implies a focus on short term and specific learning outcomes in which…the computer tends to be viewed as a device that is passive and essentially neutral in regard to specific learning outcomes” (p. 483). On the other hand, children can learn with a computer. From this perspective, the focus is on “long-term, broader, less specific, and sometimes incidental outcomes in which the computer plays an active role” (Labbo and Reinking 1999, p. 483). This perspective acknowledges the broader cognitive and social components of learning using technology. Research from this viewpoint tends to be guided by social theories of learning and focuses on the role of the technology as well as the multiple realities of combining technology and learning (Labbo and Reinking 1999). For this study, the researcher assumed that children learn with a computer.

Because of this study’s emphasis on an emergent literacy perspective, a framework based on Vygotsky’s social learning theory (1978) seemed most relevant for investigating how students interact with technology in the classroom. Vygotsky’s theory of learning assumes that both teaching and learning are highly shared and interactive activities. Specific to literacy, Vygotsky’s theory suggests that children develop understandings about language, reading, and writing through social interactions that occur with MKO. In most educational settings, teachers scaffold these interactions through the zone of proximal development (ZPD), defined as the “distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance or in collaboration with more capable peers” (Vygotsky 1978, p. 86). The concept of the ZPD or scaffolding can be extended to examine the support offered by technology; however, this is not always the prevailing view in the literacy field.

One issue specifically related to educational technology and early literacy is the inconsistency between the design of many educational technology programs and the principles of the emergent literacy paradigm. In general, most educational technology is based on behaviorist assumptions, which focus on repetition, immediate feedback, and reinforcement, rather than social learning (Johnson et al. 2010; McLoughlin and Oliver 1998; Paterson et al. 2003). The lack of pedagogical models guiding many technology applications for early literacy means that much of the technology does not align with early literacy principals that value social learning and the interaction that takes place among adults and children in a classroom. Lankshear and Knoble (2003) suggest that studying technology and literacy from a broadened perspective that includes learners, teachers, and context would provide research that more closely approximates emergent literacy theory.

Using this broadened perspective, educational technology can often serve as the facilitator or guide in what was previously a teacher-student interaction. In general, the goal of many educational technology programs is to emulate or copy the instructional methods of a human teacher (Burnett and Daniels 2015; Johnson et al. 2010). Specific applications of technology attempt to take on the role of the teacher by giving immediate feedback on responses and by providing further practice at the students’ instructional levels (McLoughlin and Oliver 1998). By giving feedback and adapting instruction based on a student’s individual needs, educational technology attempts to provide instruction in the student’s ZPD. To the degree that technology can simulate this human interaction will determine its success as a MKO and its success in producing socially created knowledge for early literacy learners. While previous research has examined the use of technology as an adaptive and intelligent tutor (e.g., Johnson and Lester 2016; Kim and Baylor 2006, 2016; Zheng 2016), the specific application of educational technology as a MKO has not been previously examined in the literature. Because of the importance of social interactions, particularly those in which teachers scaffold children’s early literacy development, this study carefully considered whether technology can be an effective mediator between the child and the social construction of literacy knowledge, serving as a MKO in a classroom.

Methods

Research design

Based on the nature of the research questions, this investigation was conducted using an embedded mixed methods approach (Teddlie and Tashakkori 2009). As Teddlie and Tashakkori (2009) note, “(Mixed-methods) research provides better (stronger) inferences (and) provides the opportunity for a greater assortment of divergent views” (p. 33). The opportunity for divergent views and multiple realities is a particularly important aspect of why this study was conducted using a mixed-methods approach, as it aligns with Labbo and Reinking’s (1999) multiple reality theoretical framework for researching technology and literacy.

The quantitative analysis measured the gains made in literacy achievement by students using Istation® and students not using Istation®. A second qualitative analysis, embedded within the first stage, collected observational and interview data on the teachers and classrooms within these twelve classrooms. The qualitative data were then analyzed and used to create teacher profiles to match participants in the study and to create an additional independent variable for further quantitative analysis. In addition, by examining teacher and classroom variables through observation and survey, the study was able to more adequately account for the contribution of various teacher and classroom variables when matching students and creating control and treatment groups (Fig. 1).

Fig. 1
figure 1

Diagram of the research design for the current study. Mixed method embedded experimental design. The qualitative data were used to create a teacher variable that was used to match participants for propensity score matching as well as a three-level independent variable for the second research question

Full size image

Because randomly assigning children to use or not use Istation® was not possible, matched control and treatment groups were constructed through the use of propensity score matching in order to control potential variation (beyond the instructional format presented) at the participant level. This approach allows for quasi-experimental comparisons between children in naturally occurring treatment and control groups. Propensity score matching is one way to mimic the random selection of participants of a randomized control trial (RCT) in an observational survey (Rosenbaum and Rubin 1983). Because of its ability to reduce selection bias, propensity score matching is increasingly being used in educational research (Graham and Kurleander 2011; Murnane and Willett 2011).

Rosenbaum and Rubin (1983) defined propensity scores as the conditional probability of treatment assignment based on certain observed baseline covariates. More simply, the propensity score is the predicted probability of treatment after accounting for important matching variables (Reutzel et al. 2012). The goal or objective for a researcher using propensity scores is to select a sequence of variables, based on theory and research that are considered important in matching participants (Reutzel et al. 2012). If the theory and history on which the researcher bases his/her selection of covariates is good, then the model is sound and causal inferences can be made (Reutzel et al. 2012; Thoemmes and Kim 2011). Once propensity scores were estimated for participants from the control and treatment groups using logistic regression, the probabilities were then used to match students who received the treatment with those who did not receive treatment (Austin 2011; Reutzel et al. 2012). By matching participants with similar propensity scores, the measured covariates were more equally distributed among the treated and control groups (Austin 2011). As Thoemmes and Kim (2011) note, “The assumption is that the matched samples of children are identical (or at least comparable) on many background characteristics and only differ in their (treatment) status—just as we would expect from a randomized experiment” (p. 93). Both theory and prior empirical research were used to identify variables that influence young children’s early literacy skills. Participants for this study were matched on the following variables: (a) Age on the first day of kindergarten (Huang and Invernizzi 2012), (b) gender (Below et al. 2010; Chatterj 2006), (c) ethnicity (Chatterj 2006), (d) free and reduced lunch status (Chatterj 2006; D’Angiulli et al. 2004; Taylor and Schatschneider 2010; Ready 2010), (e) English language learner status (Gottardo and Mueller 2009; Yesil-Dagli 2011), (f) beginning of year letter identification score (Bishop and League 2006; Schatschneider et al. 2004), and (g) level of literacy support provided by the teacher (Boonen et al. 2014; Konstantopoulous and Chung 2011).

Participants

Participants for this study were 72 students chosen from 12 kindergarten classrooms within two suburban school districts in the Southern U.S. District A is located in a medium-size suburb while District B is a located in a large suburb in the same area. The six treatment classrooms were located in three schools within District A. District A integrates Istation® into its kindergarten literacy curriculum and requires all teachers to use the program regularly. The remaining six classrooms served as a control and were located in three schools within District B. District B integrates technology regularly into the kindergarten curriculum; however, the district uses a more traditional curriculum to directly instruct students in literacy.

Selection of schools

Because of the differences in the demographic data between the two districts and in order to create a more balanced sample for matching, purposeful sampling was used to select three comparable schools in each district. One school from each district that was not classified as Title 1, one school that was classified as Title 1, and one that was both Title 1 and had a high English Language Learner (ELL) population were selected. The schools were matched as closely as possible on school size, percentage of economically disadvantaged students, ELL population, and ethnic and minority composition as shown in Table 1.

Table 1 Demographics of study participants by group
Full size table

Selection of teachers

After meeting with each the school principals, the researcher asked the principals to provide the names of two kindergarten teachers who would be willing to participate in the study. All students in the kindergarten classrooms of the teachers who volunteered were asked to participate in the study.

Student participants

The final analysis included 72 students matched through propensity score matching, with 36 students in each of the treatment and control groups.

Instrumentation/materials

DRA2

The Developmental Reading Assessment-2 (DRA2) is a widely used, criterion-referenced reading assessment for children in kindergarten through third grade (Beaver 2006). It is modeled after an informal reading assessment and uses authentic texts to measure students’ independent reading level. The DRA2 was chosen as an outcome measure for independent reading level for this study because both District A and District B already use the assessment to determine the reading levels of their students at mid-year and end-of-year.

Clay’s observation survey

To accurately reflect an emergent literacy perspective and the complexity of literacy, Clay’s Observation Survey (2002) was used to measure literacy learning. Like the DRA2, the Observation Survey of Early Literacy Achievement (OS) (Clay 2002) is an individually administered assessment tool that is widely used in early literacy classrooms in the United States and is conducted in the context of authentic literacy tasks.

Clay’s survey can be broken down into several related dependent measures. These subskills include hearing and recording sounds in dictation, writing vocabulary, letter sound knowledge, concepts about print, word reading, and reading level, for a total of six possible dependent variables; however, for this study, the reading level subtest of Clay’s survey was replaced by the DRA2.

Procedure

Project design

All students in the studied classrooms followed the district-mandated curriculum for kindergarten for the 24-week investigational period. District A, the treatment group, requires its teachers to use Istation® as part of the kindergarten curriculum, while District B, the control, does not. All of the studied schools in District A began Istation® use by the third or fourth week of school. The average time that each of the treatment participants spent on Istation® was 135 min per week. Both districts encourage an emergent literacy approach in their kindergarten classrooms, with authentic, integrated methods of instruction, including shared reading, guided reading and journal writing. The current study was conducted during the 2013–2014 school year.

Description of Istation ® treatment

The Istation® program was developed around four main components: assessment, instruction, reporting, and teacher tools. These four components are aligned and integrated into the state curriculums and are part of what makes Istation® an ILS. Istation® has aligned each of its lessons with the Common Core objectives and with the learning objectives of 42 states plus the District of Columbia and the US Virgin Islands (Istation®, n.d., Instructions: Correlations section).

Istation® begins by having students log in and take an assessment that lasts 40 min or less (Mathes et al. 2012). These assessments are called Istation® Indicators of Progress (ISIP™). ISIP™ attempt to determine students’ abilities in the five critical reading areas and are mainly multiple-choice, with a few fill-in-the-blank questions. Using item response theory and computer adaptive testing algorithms, the program adapts, varying the difficulty and number of questions depending on how the student responds (Mathes et al. 2012). ISIP™ are independent of age or grade level. Based on the assessment results, Istation® places the student within the reading curriculum.

After students are assessed, they receive systematic and explicit direct instruction and practice on their level. The instruction follows a typical lesson plan format, including an introduction, modeling, guided practice, independent practice, and an application within a book or passage. Typically, teachers took their classes daily to the computer lab and had them log into their Istation® accounts and put on headphones. Interactive activities, games, and animated characters such as Detective Dan and the Digraphs were integrated into the lessons (see Fig. 2).

Fig. 2
figure 2

A screenshot from Istation® program (Istation® 2015)

Full size image

If a student was successful during the lesson, the program adapted and moved on to the next lesson in the Istation® literacy curriculum. If the student struggled during a lesson, the program automatically adapted and retaught the skill in another format (see Fig. 3).

Fig. 3
figure 3

A figure from “technical report: Istation®’s indicators of progress: early reading version 4,” by Mathes et al. 2012. Copyright 2012 by Istation

Full size image

Baseline measure of reading achievement

Because of the diverse nature of the schools and teachers in naturalistic inquiries, it is often difficult to obtain pretest scores that can be used across participants for baseline comparisons. For this reason, letter identification was chosen as a baseline measure for this study. Letter identification is a widely used screening and assessment tool in many kindergarten classrooms. Kindergarten teachers use this easy-to-administer assessment as a way to efficiently gauge their students’ initial levels of literacy learning. All of the participating teachers collected beginning of the year letter identification data within in the first six weeks of the school year. These scores were used as a baseline measure of achievement for the propensity score matching (Fig. 3).

Controlling for teacher variables

Each kindergarten classroom was observed for a total of 4 h during literacy instruction during February 2014. Most classrooms were observed two times for half days, averaging 2 h for each observation. Observation protocols were adapted from Paterson et al.’s (2003) study on a similar integrated learning system. The following data were recorded on uniform observation worksheets: (a) Description of the classroom, (b) start/end time of activities, (c) materials used for the lesson, (d) teacher behaviors, and (e) child behaviors.

Coding of observations

The purpose of collecting the observational data was to determine the level of early literacy support provided by each of the participating teachers. Prior to conducting the classroom observations, the researcher constructed teacher profiles and a coding framework for low literacy support, medium literacy support, and high literacy support, using descriptions of effective early literacy practices from the research (Cunningham and Allington 2010; Thompkins 2014). The coding framework listed and described 15 effective literacy practices. Each of the 15 literacy practices included a detailed three-level description for low literacy support, medium literacy support, high literacy support. Using the original field notes, the researcher coded the teacher behaviors and classroom interactions as low, medium, or high according to each of the 15 literacy practices in the matrix. Based on patterns of support in the coding, an overall profile was determined for each teacher and placed the teachers into one of the three levels of literacy support, as shown in Table 2.

Table 2 Teacher profiles matrix for low, medium, and high levels of literacy support
Full size table

Intercoder agreement

To establish intercoder agreement on the observational data, the researcher asked a language and literacy doctoral candidate, who was also a certified teacher with 11 years of experience in the lower grades, to code a random sample of four observations using the coding framework. This check coding was done after the observations were completed. After a 30-min training session, the doctoral student coded the teachers using the original field notes from the observations as high, medium, or low on all 15 of the effective literacy practices and assigned each teacher an overall profile. Agreement on the overall level of literacy support provided by the teachers in the four observations was 100%.

Research memos

Shortly after each observation, the researcher created a research memo that contained reflective notes about the classroom observation and teacher behaviors. The memos noted any relevant comments the teacher made and also noted emerging patterns, insights, and connections in the observational data.

Teacher survey

The twelve participating teachers were asked to complete a survey of literacy practices adapted from a survey by Paterson et al. (2003). The survey had a checklist of 12 components commonly found in early literacy programs as well as open-ended questions. The 12 components ranged from shared reading to writer’s workshop. Teachers were asked to rate each these components on a scale from 1 to 3, based on how important the component was to their literacy curriculum. The open-ended questions on the survey asked teachers to further explain their future goals, areas of strength in literacy instruction, and differentiation strategies. The teachers who used Istation® were also asked about the best features and biggest concerns regarding the program. The list of practices the teachers identified as a critical part of their curriculum were coded as high, medium, and low literacy support using the same coding matrix developed for the observational data. The list of practices the teachers identified as a critical part of their curriculum were triangulated with the coding on the teachers’ observed literacy practices as well as the research memos to confirm the level of literacy support provided by the teachers. The individual teacher profiles provided a practical synthesis of the three qualitative data sources. The teacher profiles (high, medium, low) created from the observational data and teacher surveys were then used for two purposes: (a) as a covariate in creating propensity scores to match students for the study, and; (b) as an independent variable in the second and third research questions about the effect of this support on the literacy learning of the kindergarten students.

Measuring literacy achievement

Data on participants’ literacy achievement was collected during February 2014 from two sources:

  1. (1)

    DRA2: The twelve participating teachers provided students’ middle of the year DRA2 (MOYDRA2) scores to the researcher. This teacher-administered individual assessment was given to all participants in January 2014. This measure was used to determine participants’ independent reading levels.

  2. (2)

    Observation Survey: Two trained research assistants and the researcher individually administered five subtests of the Observation Survey to the 150 students who returned the consent forms. Each testing session averaged approximately 30 min.

Training

The two research assistants who assisted in collecting data for this study were certified teachers with master’s degrees in education and an average of 28 years of teaching experience. The assistants had backgrounds in early childhood, elementary education, English as a Second Language (ESL), special education, and speech pathology. Each of the research assistants conducted approximately a third of the Observation Surveys. Assistants were trained on the Observation Survey during a 1 h session with the researcher. Standardization of the assessment was accomplished through a detailed protocol for the order of subtests, materials, instructions during the assessment, and scoring guidelines. All assessments were scored individually and any discrepancies were reviewed and resolved according to the protocols established for the assessment.

Data analyses

To determine the effect of Istation® and the effect of the level of literacy support provided by teachers on the literacy achievement of kindergarten students, propensity score estimation was used to match students from the treatment and control groups. For this study, students who used Istation® were matched with students who did not use Istation®, using variables that both theory and research have identified as having an influence on early literacy achievement. Use of propensity score matching allowed for relatively unbiased estimates of Istation®’s causal effect on the participants’ early literacy skills, closely approximating those that could be obtained from randomized control trials (Austin 2011; Murnane and Willett 2011).

The full sample of 150 students was used to match students. In the data set, 80 of these participants were in the treatment group while 70 participants were in the control group. An initial propensity score was estimated using the seven variables derived from early literacy theory and research. Treated and untreated participants were matched using an optimal, nearest neighbor with caliper matching algorithm (Austin 2011). The caliper width used was equal to 0.2 of the standard deviation of the logit of the propensity score (Austin 2011, 2014). Research has confirmed that caliper matching leads to improved balance on baseline covariates and less bias in treatment effect estimates (Austin, 2014). When participants who used Istation® were matched with participants who did not use Istation® based on the logit of the propensity score algorithm, 36 matched pairs were formed, for a total sample of 72 participants. Once students were matched, two analyses were conducted on the data to answer the three research questions:

  1. 1.

    A descriptive discriminant analysis (DDA) (Huberty 1994) was conducted to evaluate the effect of Istation® on the early literacy skills of kindergarteners and to determine which variables contributed to any differences between the two groups;

  2. 2.

    A 2 × 3 multivariate between-subjects analysis of variance (Istation®: No/Yes X Teacher Support: Low/Medium/High) was conducted to test for main effects for level of teacher literacy support and to test for a multivariate interaction between Istation® and level of teacher literacy support.

Because the main research question focused on the effects of Istation®, the first analysis tested for the effect of Istation® on the DRA2 and the five subtests of the Observation Survey. The teacher variable discovered during the qualitative portion of this study was used to explain any possible differences based on the level of literacy support provided by the teacher.

Effects of Istation®

A DDA (Huberty 1994) was used to evaluate whether students who used Istation® and students who did not use Istation® differed in their early literacy knowledge. Table 3 reports the means and standard deviations of the two groups regarding early literacy achievement. Visual analysis of the group means indicated there were differences and that the groups would be good discriminators because the separations between the groups were moderate.

Table 3 Means and standard deviations on the six literacy concepts for Istation® versus control
Full size table

Histograms and significance tests of the data indicated a violation of the assumption of multivariate normality (z = −8.143, p = .001). For the Letter Sound Knowledge and Hearing and Recording Sounds subtests, in particular, a negatively skewed distribution was evident and univariate tests of normality showed substantial deviations from a normal distribution. To help the data meet normality and heteroscedasticity assumptions, the six dependent variables were transformed using Box-Cox procedures (Osborne 2010). Tests of the transformed data indicated that all of the variables met the assumption of multivariate normality. To determine if the data met the homogeneity of variance assumption, a Box’s M test was run on the transformed data. Box’s M, F(21,18022.2) = 1.43, p = .094, was not statistically significant, indicating that the covariance matrices for each group were approximately equal.

To determine differences in early literacy skills between the two groups, the transformed data were then analyzed using discriminate analysis in SPSS, version 22. Canonical discriminant functions are used to determine if the variance in the synthetic dependent variable can be explained by the independent variable in the model. As shown in Table 4, the canonical discriminant correlations data for this study indicate that there was a correlation between the synthetic dependent variable (early literacy skills) and the independent variable (Istation® vs. control) on Function 1 (.421) with an effect size of \({\text{R}}_{\text{c}}^{ 2}\) = 17.7%. This means that the use of Istation® was able to explain 17.7% of the variance in group differences. The full model test for

Table 4 Wilks’ Lambda and canonical correlation for two groups on Box-Cox transformed data
Full size table

Function 1 was statistically significant at p = .04.

To help determine the relevance of the dependent variables and to evaluate which of the six variables contributed to differences in the early literacy skills achievement between the groups, the researcher examined the standardized discriminant function coefficients and structure coefficients for the transformed data (Henson 2002). Table 5 combines these two sets of coefficients. Analysis of the data indicates that Hearing/Recording Sounds and Letter Sound Knowledge were the dominant contributors to the differences between groups, accounting for 35.3% of the variance. Writing Vocabulary contributed minimally to group differences. The contributions of the DRA2 and Reading Words were negligible while Concepts About Print was a suppressor in the model.

Table 5 Standardized discriminant function and structure coefficients for the transformed data
Full size table

Effect of level of teacher literacy support

To evaluate the effect of the level of teacher literacy support, a 2 × 3 (Istation®: Yes/No X Teacher Support: Low/Medium/High) multivariate between-subjects analysis of variance was conducted on the transformed data. Neither the multivariate main effect of teacher support nor the multivariate interaction was significant; however, both models were able to explain a meaningful amount of the variance between and among the groups.

The multivariate main effect of teacher support was able to explain 18% of the variance between the groups. There were significant overall group mean differences based on level of teacher support on three variables: DRA2 F (2,69) = 3.91, p = .025, η2 = .106; Concepts About Print F (2,69) = 3.60, p = .033, η2 = .098 and Reading Words, F (2, 69) = 3.232, p = .046, η2 = .089.

Table 6 reports the means and standard deviations on the non-transformed data.

Table 6 Means and standard deviations for nonsignificant teacher support main effect on non-transformed data
Full size table

Further examination of the individual group univariate statistics indicated that there were statistically significant differences between low support teachers and high support teachers on three variables: DRA2 (Reading Level), Concepts About Print, and Reading Words. Given the statistically significant F-test for each main effect, the researcher conducted post hoc analyses to further examine the group differences. Specifically, the Fischer LSD test was used on all possible pairwise contrasts. For DRA2, the following pairs of groups were statistically significant (p < 0.05): low support teachers versus high support teachers (p = 0.026), and medium support teachers versus high support teachers (p = 0.022). There was no statistically significant difference between low support teachers and medium support teachers (p = 0.503).

The overall interaction effect was able to explain 24.5% of the variance in group differences. Visual analysis and comparison of the non-transformed group means revealed meaningful patterns among the different levels of teachers depending on whether Istation® was used or not. Interestingly, participants in the control classrooms with a low support teacher had higher scores on five out of six of the dependent literacy variables when compared to their matched peers in the Istation® classrooms with low support teachers. This finding suggests that Istation® may not have been as effective with students in classrooms with low support teachers; however, it is worth noting that a wide variation in cell size existed, with a lower number of participants with low support teachers. Conversely, participants in the Istation® classrooms with a medium support teacher had higher scores on four out of the six dependent variables, with the Istation® participants scoring higher on all subtests except for the overall reading level and writing vocabulary. These findings suggest that overall, Istation® was more effective when used in classrooms with medium support teachers. Finally, analysis of the group means for high support teachers revealed interesting patterns. Participants in the Istation® classrooms scored higher on hearing and recording sounds, letter sound knowledge, writing vocabulary, and reading words while participants in the control classrooms scored higher on overall reading level and concepts about print.

Results

Summary of findings

The principal findings of this study are that:

  1. (1)

    Istation® did have a statistically significant effect on the early literacy skills of the kindergarten students studied and was able to explain 17.7% of the variance in group differences.

  2. (2)

    Differences in Hearing/Recording Sounds and Letter Sound Knowledge were the two main contributors to the variability between the two groups. Variability in Writing Vocabulary contributed minimally to the group differences while Concepts About Print was a suppressor variable in the model.

  3. (3)

    Level of teacher literacy support was able to explain 18% of the variance between the two groups. Overall, the model was not statistically significant; however, analysis of the individual group means did reveal significant group differences on Concepts About Print, Reading Words, and middle of the year DRA2 based on the level of literacy support provided by their teachers.

  4. (4)

    The interaction effect between the Istation®/control and the level of teacher literacy support was not statistically significant partly due to the low number of matched participants in some cells; however, the model was still able to explain 24.5% of the differences among students receiving different levels of literacy support from their teachers.

Discussion

By investigating technology from an early literacy perspective, multiple realities about the complex relationship between technology and early literacy learning were uncovered. Results indicate that overall, Istation® had a statistically significant effect on the literacy learning of the students in the study given the dependent measures used. This study also considered the effects of teacher support in early literacy classrooms, independent of the Istation® program. Finally, this study analyzed patterns in the interaction between the use of Istation® and teacher support. By considering these results together, the researcher can speculate as to why the Istation® program and the level of literacy support provided by teachers had an effect on particular measures of early literacy skills and comment on the role that Istation® should play in early literacy classrooms.

Based on the results from this study, Istation® was particularly effective in developing students’ letter sound knowledge and their ability to hear and record sounds. Istation®’s approach to instructing students in early reading aligns closely with many behavioral theories of learning and reinforces those reading skills (like letter sound knowledge) that require speed and efficiency (Ehri and Roberts 2006; Ertmer and Newby 1993; NELP 2008; Philips and Torgesen 2006; Skinner 1954). Istation® encourages automaticity through multiple opportunities to practice skills using highly structured, individualized instruction and by creating strong connections between stimulus and response through reinforcement. Despite a significant overall effect, Istation®’s approach to instruction did not seem to be as effective with what are arguably more complex measures of early literacy such as overall reading level and concepts about print. In the case of these skills, the level of literacy support provided by the teachers had a larger effect.

The data suggest that as the level of literacy support provided by the teacher increased, students’ abilities to read and comprehend a book, understand concepts about print, and read novel words also increased. While Istation® provided multiple opportunities for practice and feedback on students’ individual levels, the more constructivist approach of the teachers in this study allowed for greater social interaction, small group instruction, and flexibility in instruction and student products. The main goal of many of the teachers did not appear to be speed and accuracy; rather, the emphasis was on modeling the flexible application of strategies and allowing students to practice these strategies within a social context. This shared and interactive approach to literacy instruction closely aligns with emergent literacy theory and research and would explain why teachers had a greater effect on literacy measures that require a greater depth of processing and a more flexible application of strategies (Blair et al. 2007; Clay 1991; Hall 2003; NELP 2008; Schunk 1991; Sulzby and Teale 1991)

A case could be made that, by definition, Istation® modeled the instruction of a MKO, guiding and scaffolding instruction as students practiced new skills. Istation® provided feedback, evaluated students’ responses, and adapted instruction based on students’ responses; however, based on observations, there are several notable differences between the way that Istation® modeled and supported literacy learning and the ways in which the teachers did. These differences help explain why Istation® did not have a broader effect on students’ early literacy achievement.

The most significant difference between Istation® and the teachers was the authenticity of the early literacy experiences observed. During Istation® instruction, students wore headphones, were generally quiet, and did not interact with the teacher or each other. Students did not produce or respond verbally to any texts—they simply clicked on the right answer when prompted by the program. There was little variability in the presentation of the content. On the other hand, during large and small group instruction with the teachers, students interacted socially, responded to texts in a variety of ways (including writing), and there were a variety of texts and contexts presented. Early literacy research supports social interaction and variability during instruction (Blair et al. 2007; Hall 2003; NELP 2008; Ponitz and Rimm-Kaufman 2011).

Another notable difference was the flexibility and adaptability of Istation® compared to the teachers. As an adaptive sequence system, Istation® is based on the concept of mastery learning. Istation® adapted to individual students based on their responses; however, the program was not as flexible or responsive as the classroom teachers were during similar activities. For example, many of the teachers were observed quickly adapting instruction based on individual students’ interests, backgrounds, and specific needs. The teachers often revised their approach based on social interactions that occurred during instruction. One adaptive strategy that many of the teachers used that Istation® did not was metacognitive instruction. The teachers explained, modeled, and used reading strategies during instruction, thinking aloud as they did so. This metacognitive layer of instruction is vital to creating strategic readers (Afflerbach et al. 2008; Ankrum et al. 2014; Pressley et al. 1998).

Finally, the researcher observed different emotions from students as they interacted with Istation® compared to the emotions as they interacted with their teachers. As Cambourne (1995) asserts, children are more successful when their learning is supported by “those to whom they are bonded” (p. 185). Students appeared motivated to work on Istation® and were generally engaged with the program; however, there was no “bond” observed between the students and the computer. On the other hand, students often seemed emotionally connected to their teachers, and they were observed smiling, laughing, and socially engaged during instruction. These strong emotions and bonds during instruction have been shown to lead to increased learning in students (Jensen 1998; Ponitz and Rimm-Kaufman 2011; Wolfe 2010).

Conclusion

One purpose of this study was to determine if Istation® was able to serve as a MKO. Based on the findings of this study, it depends. Istation® appears to effectively instruct young students and serve as a MKO within some aspects of literacy instruction, particularly those that involve early literacy concepts that require drill and repeated practice, such as letter sound knowledge, hearing and recording sounds, and writing vocabulary. For these early literacy skills, Istation® was able to scaffold students’ learning, provide instruction within their zone of proximal development, and serve as an effective mediator (or MKO) between the child and the social construction of early literacy knowledge.

In contrast, Istation® does not appear to be an adequate substitute for the MKO when it comes to creating meaning and applying early literacy skills to more complex literacy tasks. Based on the data from this study, early literacy skills that require the integration of a variety of literacy skills and strategies, such as reading and comprehending a book, understanding concepts about print, and reading words, seem to require the instruction and feedback of a human, one who is able to interact, provide multidimensional feedback and allow for the student to take on a more active role in the social interaction.

There has been much debate recently about whether technology will replace teachers in the classroom. More and more, research on educational technology is providing evidence that a teacher’s role in the classroom is strengthened by the introduction of new technologies. This study provides further evidence that technology supplements, not supplants the teacher in the classroom.

Limitations

The results of this study were limited by the total sample size (n = 72). Despite attempts to use create a more balanced sample through purposeful sampling of schools, only 72 of the 150 participants were able to be matched using propensity score matching. In addition, propensity score matching assumes unconfoundedness and assumes that no further variables exist that may predict the propensity of the participants (Austin 2011; Murnane and Willett 2011; Reutzel et al. 2012). Because of this assumption, the design of this study may be limited by the matching variables selected by the researcher.

This study is also limited by the geographical location and demographics. The study took place in two medium to large, diverse, suburban school districts in the south, and the results may not be generalizable to other regions or school districts.

Implications for use of Istation® in early literacy education

There is increasing pressure on school districts to find quick and efficient solutions to perceived problems in reading achievement, and often, the focus is on improving early reading skills (Paterson et al. 2003). A popular solution to these problems is educational technology. As the use of technology becomes more prevalent in elementary schools, and particularly in early childhood classrooms, there is an increased need for independent research on the relationship between technology and literacy in order to justify (or discourage) districts’ large expenditures and inform their decisions about how to integrate technology into the instructional curriculum (Tracey and Young 2007). This study contributes to the scant literature on the effects of technology on the early literacy skills of young students and provides emerging evidence supporting the use of integrated learning systems, including Istation®, as one tool in the early literacy curriculum.

Implications for future research

To increase the ability to generalize findings, future researchers may want to use a larger sample size, include other grade levels, select different types of school districts, or study specific populations (ELLs, economically disadvantaged, struggling readers). In addition, a study on the qualitative differences between the instruction and feedback of Istation® versus the teacher would be helpful in further evaluating when and how technology can be integrated effectively into the curriculum.

Despite the limitations and suggestions for improvement, this study provides important evidence supporting the efficacy of Istation® with a small sample of kindergarten students. When integrated into a curriculum in which teachers support literacy learning through a constructive approach, Istation® can offer teachers and districts a potentially efficient and effective tool for providing some of the early literacy instruction for young students.