Introduction

Decades of research demonstrate how positive social skills and relationships are associated with children’s positive behavioral, emotional, and academic well-being (Blair 2002; Connell and Prinz 2002; DeRosier 2004; Eisenberg et al. 2005). The ability to establish and maintain positive relationships with peers is a key developmental milestone and has been associated with higher self-esteem and self-confidence (Buhrmester 1990; Guralnick 1993; Hemmeter et al. 2006; Nelson and Aboud 1985). However, a significant number of children experience difficulties interacting with their peers, resulting in bullying, social isolation, and peer rejection (Buhs et al. 2006; Campbell et al. 2000; Johnson et al. 2000). Without intervention, children who struggle in their social relationships often face a wide range of later difficulties, including school dropout, drug abuse, depression, and antisocial behavior (Howes 2000; Keane and Calkins 2004; Ladd 1990; Johnson et al. 2000; Snyder 2001; Tremblay et al. 1996).

Given the substantial evidence of the negative consequences of peer problems and, conversely, the buffering effects of positive peer relations, efficient delivery of effective social skills training (SST) strategies could have a broad impact on children’s social, emotional, and behavioral health, as well as their academic success. In general, SST applies a collection of cognitive and behavioral approaches to teaching age-appropriate social skills and competencies, such as communication, problem solving, decision making, self-management, and making and keeping friends. Previous research has shown wide applications for in-person SST programs (e.g., a trained provider implementing a program within small groups of 6–8 children), from assisting young children with developing self-awareness and emotion regulation skills (Moreira et al. 2010) to helping children with attention problems learn better self-management strategies (Raimundo et al. 2013) to promoting social problem-solving skills and adaptive social behavior among children with autism spectrum disorder (Mathews et al. 2013; Reichow and Volkmar 2010; Scattone et al. 2006). In addition to its use with specific populations of children, research supports the benefits of SST as a universal, integrated intervention approach by which all children participate in social emotional learning, regardless of diagnosis or special education status, and thereby all children can benefit from enhanced social skills and a more positive overall school climate (e.g., Jones et al. 2010).

While the benefits of in-person SST programs are well documented, this mode of delivery presents a variety of problems for schools, parents, and providers. Logistical barriers, particularly financial costs to families and schools, lack of sufficient numbers of trained providers, and significant time, travel, and scheduling requirements, prevent many children from participating in, and benefitting from, SST (Mueser and Bellack 2007). Beyond these practical challenges, the efficacy of traditional SST methods can suffer if providers implement the program inconsistently or deviate from the intended design (Gresham 2004; Leff et al. 2001). High-quality implementation is crucial to intervention success and the fidelity with which the SST is implemented has been shown to impact child outcomes (Dane and Schneider 1998; Mihalic 2004).

In an effort to more broadly deliver effective SST, intervention developers are beginning to employ emerging technologies, such as virtual game-based learning environments. These software programs offer a number of advantages over traditional in-person delivery methods. First, virtual SST is a more cost-efficient alternative, eliminating travel and trained provider costs and allowing for greater schedule flexibility while simultaneously increasing practice opportunities for children. Second, games can deliver SST in a standardized fashion, eliminating the effect of human error on implementation, thereby ensuring the program is consistently delivered with high fidelity. Third, games leverage children’s natural propensity for technology to provide a platform that may increase their motivation and engagement in SST programs (Fitzgerald 2005; NCREL and Metiri Group 2003).

Perhaps the most important advantage, however, is that computerized learning environments can easily collect and analyze performance and engagement data without interruption of services by recording children’s behaviors and choices as they engage with the software (Shute et al. 2009). This capability is particularly important given current educational trends towards Response to Intervention (RtI) models in which ongoing data collection is a central component. The National Education Technology Plan (U.S. Department of Education 2010) emphasizes that “technology-based learning and assessment systems will be pivotal in improving student learning and generating data that can be used to continuously improve the education system at all levels” (p. 5). Further, games that employ intelligent tutoring systems (ITS) are able to dynamically respond to user input during the course of gameplay (e.g., raising or lowering pedagogical assistance based on children’s performance) (Psotka and Mutter 1988). This dynamic responsiveness to measureable performance data offers individualized gameplay based on the child’s unique interactions with the software to provide a personalized learning experience through virtual situations that are analogous to those in real life (Gee 2003; Perotta et al. 2013).

While ITS software programs have prospered in a variety of academic domains over the last decade (Graesser et al. 2011; Graesser 2013), including basic mathematics (e.g., AnimalWatch: Beal et al. 2010), reading (e.g., READ 180: Haslam et al. 2006), and statistics (e.g., Web Interface for Statistics Education: Aberson et al. 2003), they are rare in the area of social emotional learning. A few virtual SST programs have been recently developed (e.g., Adventures Aboard the S.S.GRIN: Sanchez et al. 2014; Quest for the Golden Rule: Rubin-Vaughan et al. 2011) with preliminary results supporting the potential of ITS implementation of SST with children. Evidence suggests adaptive games can be an effective route to improving prosocial knowledge and behavior, including awareness of how to cope with bullies and the respectful treatment of friends, as well as greater self-control and appropriate timing when engaging with peers (Fenstermacher et al. 2006; Rubin-Vaughan et al. 2011; Sanchez et al. 2014).

Drawing on the foundational principals of ITS software and the emerging virtual SSTs for children, we developed Zoo U, an intelligent social tutoring system (ISTS) designed to both assess and build prosocial skills for all children ages 7- to 12-years across six key social skill areas: impulse control, communication, cooperation, social initiation, empathy, and emotion regulation (DeRosier 2014; www.zoougame.com). Zoo U is a social skills intervention disguised as a computer game wherein children interact with virtual teachers and classmates to learn about and take care of animals. As children progress through the program, they receive personalized feedback about their performance from an in-game pedagogical assistant, Principal Wild, who provides skill-specific instruction relevant to what the children encountered in the game as well as explanations about why their choices were or were not successful.

Unlike other technology-based SSTs, which focus on specific populations (e.g., O’Connor et al. 2006; Tsang and Pearson 2001), Zoo U was developed to be used with any child, targeting social skill training needs that are broadly applicable to elementary-age children. By harnessing the power of automated data collection, the software is able to tailor the social learning environment to provide personalized instruction tailored to the unique SST needs of the child. When a child engages with Zoo U, s/he first completes six assessment scenes to identify social skill strengths and weaknesses and then Zoo U dynamically adapts gameplay in real time as the child interacts with the software. For example, responses by in-game characters (e.g., for pedagogical assistance) are customized to the specific choices made by the child as s/he progresses and difficulty level of subsequent scenes is adjusted based on the child’s past performance.

Progress through the Zoo U intervention is built on a mastery model for which children must meet a pre-set performance criterion for a scene in order to advance (DeRosier 2011, 2014). This mastery model encourages repeated practice in the areas a child is struggling with most and evidence suggests greater practice opportunities and repeated exposure to social concepts fosters greater treatment benefits (Foster and Bussman 2007; Greenberg et al. 2001). This mastery model is also in line with the positive behavioral intervention and supports (PBIS) framework of effective intervention practices aimed at establishing and improving the social culture, educational environment, and individual behavior of all children (Eber 2006; Sugai et al. 2000).

In the current study, we investigated the efficacy of the Zoo U ISTS using a randomized-control design. Zoo U was designed to teach six core social skills in order to improve children’s social behaviors with peers, as well as their social literacy, social self-efficacy, and peer relations. Therefore, we predicted that children who participated in Zoo U, as compared to children in the wait-list control, would show evidence of greater improvements in each of these areas. Specifically, we hypothesized that children who completed Zoo U would demonstrate significantly greater improvements, compared to the control group, in (1) parents’ reports of their child’s social skills and social behavior, (2) children’s reports of their social self-efficacy and social satisfaction with peers, and (3) children’s knowledge or social literacy in the targeted social skill areas.

Method

Participants

Participants were recruited nationally via postings on parenting and educational listservs and social media sites. Interested parents were instructed to complete an online questionnaire, which included demographic questions and the parent rating scales of the Behavior Assessment System for Children, Second Edition (BASC-2; Reynolds and Kamphaus 2004). The BASC-2 is a developmentally sensitive rating scale measure that is widely used for screening behavioral and emotional problems among youth. BASC-2 items describe specific behaviors and parents rate how frequently their child exhibits those behaviors on a four-point scale from “Never” to “Almost Always.” To participate in this study, children had to: (1) be between the ages of 7 and 11 years at the start of the trial, (2) be English-language proficient, and (3) have access to an Internet-enabled computer.

Fifty-nine eligible participants were stratified by BASC-2 subscale scores, age, sex, and race, and then randomly assigned to either the treatment condition (TX) or the wait-list control condition (CO). Children in the TX condition completed the Zoo U game over a 10-week period while children in the CO condition did not have access to Zoo U until the study trial period was over. Twenty percent of participants failed to respond to requests to complete posttests, resulting in a final sample of 47 participants with complete data (TX = 23, CO = 24). To test for possible selective attrition, we conducted Chi square analyses to compare participants who completed the posttests with those who did not on demographic variables and baseline BASC-2 adjustment. No significant differences were found, indicating that attrition was random across both conditions.

The final sample of children ranged in age from 7 to 11 years (M = 9.65; SD = 1.27) with 59 % male and 28 % representing a racial or ethnic minority group. There were no significant differences in demographic variables across the two conditions. With regard to the BASC-2, 19 % of children scored above the clinical cutoff for social problems (i.e., social skills or atypicality subscales); 20 % scored above the clinical cutoff for externalizing behavior problems (i.e., aggression, conduct problems, attention problems, or hyperactivity subscales); and 27 % scored above the clinical cutoff for internalizing behavior problems (i.e., anxiety, depression, or withdrawal subscales). Chi square analyses revealed no significant differences in these percentages across conditions, indicating similar levels of social and behavioral adjustment across the two groups at baseline.

Procedure

Prior to consenting (parents) and assenting (children) procedures, eligible parents and children viewed an online orientation video introducing them to the study design and procedures, and training them in the use of the project website through which they accessed all study materials. During the course of the trial, children completed a total of 30 Zoo U skill-building scenes. As Table 1 displays, Zoo U’s skill-building scenes are arranged into six units with five scenes of increasing difficulty within each unit. Parents of children in the TX condition were notified via e-mail and through the project website when a unit was available for completion. Children were given 2 weeks to complete all six scenes within the assigned unit. In keeping with the mastery model design of Zoo U, the scenes within each unit were delivered in progression; in order to advance, children had to either meet mastery criterion or play each scene three times, whichever came first. Children took, on an average, 8.84 min (SD = 7.52 min) to complete each scene and an average of 8.84 h (SD = 7.52 h) to complete the entirety of the Zoo U intervention. Upon completion of each unit, children completed an online quiz assessing their understanding of the instructional content presented within that unit (ALQ; see below). After completion of the unit quiz, children were given free access to the unlocked Zoo U scenes. However, inspection of the number of times children completed scenes beyond meeting mastery or completing the alternative required three attempts revealed that on average children only played scenes 2.07 times (SD = .77), suggesting that generally they weren’t playing beyond the requirements of the intervention.

Table 1 Zoo U’s learning objectives by social skill area and unit level
Full size table

Parents and children in both conditions completed surveys in the 2 weeks prior to the intervention period (pretest) and within 2 weeks following the intervention period (posttest). Children in the CO condition completed the Achieved Learning Questionnaire (ALQ) at posttest. One week following posttest data collection, children in the CO condition received access to the Zoo U program.

Measures

Parent Measure

The Social Skills Behavior Inventory (SSBI; DeRosier 2011) is a 72-item, parent-report measure designed to assess children’s social skills across ten subscales: communication, cooperation, empathy, initiation, impulse control, and emotion regulation, assertiveness, externalizing behavior problems, and internalizing behavior problems. Parents were asked to report how true statements such as, “Is aggressive towards people or objects,” “Takes turns in conversation,” and “Follows directions well” were for their child on a 5-point Likert-type scale from 1 (never) to 5 (almost always). The mean for each subscale was used for analyses, with higher scores indicating better performance on each social skill subscale, and greater reported problems for the two behavior problem subscales. The SSBI demonstrated good internal reliability for the full participant sample [overall: α = .86; subscales ranged from .60 (externalizing) to .83 (cooperation)].

Child Measures

The Self-Efficacy Scale (SES; Ollendick and Schmidt 1987) was used to assess children’s social self-efficacy. This 10-item measure asks children to rate on a 5-point Likert-type scale from 1 (not at all) to 5 (really sure) how confident they are that they can engage in 10 specific social behaviors, such as “talking with a kid your age who you just met.” The mean across items was used for analyses, with higher scores indicating greater social self-efficacy. The SES demonstrated acceptable internal reliability with this sample (α = .70).

The Loneliness and Social Dissatisfaction Scale (LSDS; Cassidy and Asher 1992) is a 19-item measure of children’s self-reported feelings of loneliness and dissatisfaction with peer relationships. Children were asked to rate on a 4-point scale from 1 (not at all) to 4 (almost always) how true statements such as “It’s easy for me to make friends” and “I feel left out of play and other activities” were for them. The mean across items was used for analyses, with higher scores indicating greater social satisfaction (and less loneliness). The LSDS demonstrated good internal reliability with this sample (α = .85).

The Achieved Learning Questionnaire (ALQ) is a 36-item measure of children’s social literacy across the six domains targeted in Zoo U (impulse control, communication, cooperation, social initiation, empathy, and emotion regulation). Children were asked to demonstrate their understanding of these social skills by answering a combination of multiple choice and true/false questions, such as, “How you say something is just as important as what you say” (true/false). The percentage of correct responses was calculated with higher percentages indicating greater social skills knowledge. In this study, the range of correct scores was 38–97 % with a mean score of 85 % (SD = 11 %). The ALQ demonstrated good internal reliability with this sample (α = .74).

Data Analyses

Before exploring the primary hypotheses of the current study, we conducted a Multivariate Analysis of Variance (MANOVA) with condition as the between-subjects factor to determine whether children’s scores for the parent- and child-report outcome measures differed at baseline (i.e., pre-intervention) for the two conditions. Results indicated that children’s adjustment prior to the trial period was statistically insignificant between the two groups, F(8, 50) = 1.93, p = .08, η2 = .24. However, given our relatively small sample size and the relatively small p value, we elected to control for baseline scores in subsequent analyses examining intervention effects across groups (Maxwell et al. 1991; Vickers and Altman 2001).

Results

For the parent-reported outcomes, Analyses of Covariance (ANCOVAs) were conducted to compare the TX and CO groups on each subscale of the post-intervention report of the SSBI, with pre-intervention reports included as a covariate. Analyses revealed significant between-group differences in parents’ post-intervention reports of children’s social initiation, impulse control, emotion regulation, internalizing behaviors, externalizing behaviors, and assertiveness. Parents in the TX group reported their children showed improved social skills in the areas of greater impulse control and emotion regulation and more positive social initiation after receiving Zoo U compared to parent-reports for children in the CO group. No significant between-group differences were found for the social skill areas of communication, cooperation, or empathy. With regard to social behaviors, whereas parents in the CO group reported slight worsening in the areas of externalizing behavior problems and assertiveness, parents in the TX group reported greater assertion skills and lower externalizing behavior for their children. Somewhat surprisingly, parents’ ratings of internalizing behavior were relatively higher at post-intervention for both groups, but particularly so for children in the TX group . Table 2 displays relevant statistics and effect sizes for these analyses.

Table 2 Summary of relevant descriptive information, ANCOVA statistics, and effect sizes for children’s social skills and behaviors via parents’ reports on the Social Skills Behavior Inventory
Full size table

For the child-reported outcomes, first ANCOVA analyses were conducted to compare children’s post-intervention reports for social self-efficacy and social satisfaction across the two conditions, with children’s pre-intervention reports included as a covariate. Analyses revealed significant differences in children’s post-intervention reports for both measures by treatment group. As expected, children in the TX group reported greater feelings of self-efficacy and social satisfaction at post-intervention compared to children in the CO group after accounting for their pre-intervention reports. Second, an ANOVA was conducted to compare the TX and CO groups’ social literacy as assessed by the ALQ. Analyses revealed a significant between-group difference in children’s overall social skills knowledge. As expected, children in the TX group showed greater social literacy. Table 3 displays relevant statistics and effect sizes for these analyses of the child-report outcomes.

Table 3 Summary of relevant descriptive information, statistics, and effect sizes for child-reported outcomes
Full size table

In an effort to explore whether the observed changes in children’s perceived social self-efficacy and social satisfaction over the course of the intervention trial were related to their level of social skills knowledge on the ALQ, we conducted follow-up correlational analyses by condition. Interestingly, correlations between the ALQ and these child outcomes were stronger for the TX group compared to the CO group. Whereas the relation between social literacy and changes in self-efficacy and social satisfaction were essentially zero for children in the CO condition (r = .01, p = .97 and r = .06, p = .80, respectively), greater social literacy for children in the TX group was significantly related to improved social satisfaction (r = .42, p < .05) and marginally related to improved self-efficacy (r = .35, p = .10). While these are only exploratory analyses, findings suggest that improvements in social skills literacy through participation in Zoo U were linked to improvements in children’s social self-perceptions.

While our sample size was too small to test whether children’s responsiveness to the Zoo U intervention differed by gender or grade level, we were interested in exploring whether there was evidence of differential treatment response by these demographic variables. This study included a range in children’s grades (2nd through 5th), such that it would be reasonable to expect that developmental maturity may have impacted the children’s ability to understand and, thus, apply the information in Zoo U to their real-world social behaviors. In addition, gender differences in children’s social skills (Caprara et al. 2001; Fabes and Eisenberg 1998; Romer et al. 2011), as well as the ways in which gender impacts parents’ perceptions of their child’s social skills (Jacobs and Eccles 1992; Lytton and Romney 1991; Schroeder et al. 2010; Webster-Stratton 1996) may have influenced the pattern of results. For the TX group, MANOVAs were conducted separately for grade and gender predicting changes in parent- and child-reported outcome measures between pre- and post-intervention. However, contrary to hypotheses, these analyses revealed no significant grade, F(24, 35) = .861, p = .645, η2 = .359, or gender, F(8, 14) = 1.84, p = .153, η2 = .512, differences in intervention effectiveness.

Discussion

We know that social skills training (SST) is an effective way to help children learn and practice essential social skills. We also know that too few children are able to benefit from in-person SST due to various logistical and opportunity barriers. It can be hard for parents to find providers who are trained in proven SST models, particularly in rural areas of the country. And even when these services are available, many families cannot afford the time or money that participation for their child would require. Given the wealth of evidence regarding how social and emotional skills help children succeed interpersonally, and how positive social relationships are critical for children’s emotional and behavioral health, greater access to effective SST could have broad reaching social impact. With recent advances in adaptive technologies, we have the opportunity to scale social emotional learning strategies through engaging, easily accessible game-based platforms. However, as with in-person treatment, it is essential that these SEL games be tested to demonstrate they are in fact beneficial for children.

The purpose of this study was to provide an initial evaluation of Zoo U as a first step in understanding how this game-based approach to SST can effectively improve children’s prosocial behaviors, social self-perceptions, and social literacy. As hypothesized, children who participated in Zoo U over the 10-week intervention trial—as compared to children who did not participate in Zoo U—showed significant improvements in both social emotional skills and social behavior. Specifically, after completing Zoo U, parents in the TX condition reported their children demonstrated significantly greater impulse control, emotion regulation, and social initiation skills, with a trend-level increase in cooperation. Further, parents reported significant changes in their child’s ability to manage interpersonal situations that involve disagreements or conflict. Specifically, children who completed Zoo U increased their use of appropriately assertive behaviors and concurrently reduced their use of antisocial and aggressive behaviors. Together, this pattern of results reflects how Zoo U can help children learn to better control their emotional and behavioral impulses, skills that are closely tied to children’s ability to form friendships, cooperate with others, and appropriately manage conflict.

Unexpectedly, parents in the TX condition reported an increase in their child’s internalizing behaviors, such as social withdrawal and anxiety, following completion of Zoo U. While surprising in some regards, this result may actually reflect a change in parent perceptions; prior research indicates that as children’s externalizing behaviors decline and they struggle to apply prosocial behaviors to their peer interactions, parents may perceive them to be more anxious or shy (Eisenberg et al. 2009). Also unexpectedly, TX parents did not report significant improvements in the areas of communication and empathy. These social skills are particularly complex and sophisticated, involving higher-order processing and an ability to coordinate multiple social behaviors at the same time (Wellman et al. 2001; White et al. 2007). It may be that the short duration of the intervention period for this study, and therefore the lower dosage of the Zoo U intervention for children, was too brief to demonstrate significant change in these more complex skills. It may also be that, as children gain experience applying the more basic social skills of impulse control, emotion regulation, and social initiation in their social lives, significant improvements in the areas of communication and empathy may emerge. Important areas for future research will be to examine varying doses of Zoo U for skill development in particular areas as well as the longer-term impact of participating in this intervention for children’s social skill development beyond the pre/post comparison.

The efficacy of Zoo U for increasing children’s knowledge and understanding of social skills was also supported through this study. As expected, children who participated in Zoo U scored significantly higher on the achieved learning questionnaire (ALQ) than did children who did not participate in Zoo U. However, social literacy alone is not sufficient for improving children’s social performance. Social self-efficacy or self-confidence is essential for translating social skills knowledge learned through SST into improved social behaviors in the real-world (Ollendick and Schmidt 1987). Importantly, the findings of this study supported significant gains in social self-efficacy and feelings of social satisfaction with peers for children who completed Zoo U. Further, significant correlations between improvements in social self-perceptions and social literacy on the ALQ were found only for the TX group. This pattern of findings suggests that the Zoo U SST program is efficacious for increasing children’s social literacy and bolstering their sense of self, thereby laying the necessary foundation for successful social relationships.

Limitations

While the randomized control design of this initial test of Zoo U provides strong evidence supporting its efficacy as a game-based SST program, a number of limitations deserve mention. First, the sample size was relatively small. Although our ability to detect statistical differences between treatment and control groups with this small sample underscores Zoo U’s potential for impacting children’s social functioning, a larger sample would engender greater confidence in the results and would likely reveal smaller yet important impacts of the Zoo U program that we were unable to detect. Further, employing larger sample sizes would allow us to investigate sub-group differences and explore mediational models that may impact Zoo U’s efficacy (e.g., impact on specific social behaviors, gender and grade differences, intervention dosage). Exploratory analyses indicated children’s responsiveness to the Zoo U intervention was consistent across grade levels and genders, but future research involving a larger sample will be needed to adequately test this hypothesis.

Another limitation of this study was reliance on child- and parent-report outcome measures of children’s social skills and social functioning. Inclusion of observational methods and additional reporters who are blind to treatment condition, such as teachers, would strengthen our understanding of how Zoo U impacts real-world behavior change and reduce concerns that results are influenced by the reporter’s knowledge that the child is receiving an intervention. Future randomized control trials using a multi-tiered approach to outcome measurement will significantly extend the findings from this preliminary test.

Lastly, the current study examined Zoo U’s effectiveness as implemented in the home. However, Zoo U is intended to be used both in schools and at home. An important next step will be to explore Zoo U’s efficacy when administered in the school setting and provide a comparison of the impact of this game-based SST for children’s social relationships in the home versus school environments. In a similar vein, it would be useful to investigate how the impact of Zoo U compares to more traditional in-person approaches to SST as delivered in school and community settings. This research would enable us to gauge the degree to which Zoo U is able to provide comparable child outcomes with fewer required resources, as well as inform how Zoo U could be used to augment traditional SST approaches to enhance overall effectiveness for improving children’s social skills and social functioning.

Conclusion

The results of this study provide initial evidence supporting Zoo U as an efficacious game-based SST program. Findings indicate that participation in Zoo U results in significant improvements in children’s knowledge of social skills, their confidence to use this knowledge in their relationships, and their ability to behave in more prosocial and adaptive ways in the real world. Both parents and children reported meaningful improvements in children’s social skills and behavior and children reported feeling more satisfied and confident in their social relationships. Overall, this study provides a strong foundation upon which future research on this novel game-based SST program can be built.

These data also contribute to a growing body of research examining computer-assisted instruction as an alternative means of providing effective social skills training (e.g., Sanchez et al. 2014). Adaptive games offer unparalleled capabilities to create personalized, dynamic, and engaging SST tools that can be delivered at low cost to youth across the globe. By lowering time, training, and financial barriers—while simultaneously maximizing engagement for children—games can be used to implement SST on a broad scale, thereby optimizing our ability to help children with social skills deficits and positively impact children’s social health and well-being more generally.