Introduction

Children who exhibit behavioral problems at an early age are at increased risk of maturing with an escalating trajectory of antisocial or oppositional behavior through adolescence and into adulthood (Biglan et al. 2004; Patterson et al. 1992). As they become adolescents, oppositional children are progressively more likely to associate with deviant peers and be involved with substance abuse, early and unprotected sex, inpatient and outpatient mental health care, and contact with the criminal justice system (Biglan et al. 2004; Dishion and Kavanagh 2003; Forgatch et al. 2009; Patterson et al. 2010). Furthermore, oppositional children are much more likely than other children to become victims of crime, fail in school, experience both peer and parent rejection, and experience unhappy, unproductive lives as adults (Kumpfer and Alvarado 2003; Patterson et al. 1992). Symptoms of oppositional/antisocial behavior are highly correlated with serious psychological, social, and economic costs for the child, the family, and the community at large (Forgatch et al. 2009; Patterson et al. 2010).

Coercive parenting practices, inconsistent discipline, and poor parental monitoring are often tied to problematic child behaviors, and improved parenting practices lead to decreases in child problem behaviors (Biglan et al. 2004; Dishion and Kavanagh 2003; Forgatch et al. 2009; Prinz et al. 2009). Problem behaviors are best interrupted early in life when behavioral patterns are more easily modified (Tremblay 2006). Support for youth can be best accomplished by supporting the adults in the family and ensuring that those adults have the necessary skills and personal efficacy to guide, support, and protect their children (Huebner and Mancini 2008).

Behavioral parent training (BPT) in a support group format has emerged as an efficacious treatment modality to improve the competence of parents whose children display oppositional and antisocial behaviors (Dretzke et al. 2009; Kaminski et al. 2008; Sanders et al. 2012; Webster-Stratton et al. 2011). The group sessions, usually lasting 8–12 weeks, are guided by therapists or therapist-trained facilitators.

Effective BPT is often associated with a sequence of positive parent behaviors linked to a decrease in negative child behaviors, which are in turn linked to positive behavior changes in other family members (Patterson et al. 2010). BPT has been demonstrated to be effective with both mothers and fathers, abusive parents, multi-distressed parents, low-income parents, minority parents, neglectful parents, substance-abusing parents, and parents of chronic delinquents (Biglan et al. 2004). Several different group-based formats have been shown to be effective, including Parent Management Training—Oregon (Forgatch et al. 2009; Patterson et al. 2010), Triple P Program (Prinz et al. 2009; Sanders et al. 2012), and Webster-Stratton’s Incredible Years (Webster-Stratton and Reid 2010; Webster-Stratton et al. 2011). Common elements include instruction, modeling positive parenting behaviors through videos and role plays, supportive group discussions, and home practice assignments to improve parenting skills. Topics in effective BPT courses include parental monitoring, effective communication strategies, positive and negative contingencies, consistent rule enforcement, and problem solving. The relative efficacy of different BPT curricula has not been determined (Dretzke et al. 2009).

Although BPT programs have been found effective, recruitment of parents can be difficult, especially in rural areas, and attrition from group sessions can be high due to personal, logistic, or cultural barriers to participation (Ouellette and Wilkerson 2008). A lack of qualified group leaders might limit the availability of BPT groups (Irvine et al. 1996).

Self-administered BPT is potentially appealing because it is available at the convenience of the user, which could decrease attrition, and efficacy does not depend on the skills of individual group leaders. Also, in contrast to group sessions, users are free to spend more time on personally relevant or interesting instructional content, and they may return to it as desired. Self-administered BPT has been shown to be efficacious in the form of text materials and modeling video tapes supplemented by telephonic or in-person coaching (Markie-Dadds and Sanders 2006; Webster-Stratton and Reid 2010) and multimedia CD-ROMs with workbooks (Cefai et al. 2010; Gordon and Stanar 2003; Segal et al. 2003). One study reported that self-administered BPT could be as effective as a group intervention led by therapists (Webster-Stratton et al. 1988). Reports differ about whether parents prefer the personal contacts available in group settings (Webster-Stratton et al. 1988) or self-directed education (Cefai et al. 2010; Gordon and Stanar 2003).

Stand-alone BPT on the Internet is a logical technological extension of self-administered programs, and it is appealing because of the wide reach of the Web, with dissemination potentially possible to millions of families. In the US, 74 % of whites, 64 % of African-Americans, and 53 % of Hispanics are now broadband users (and the numbers are rising), making online video a viable option (Brenner and Rainie 2013). Online BPT can seamlessly combine what was previously delivered via manuals and video tapes or CD-ROM into an engaging interactive multimedia experience. The self-paced nature of online learning with optional full narration of text elements might improve interest and benefit parents with poor reading or language skills. If a program is thoughtfully designed, keyboarding is unnecessary because users can mouse-click on-screen buttons to navigate or respond.

Some reports suggest that users might prefer an automated parenting program, which could be perceived as less judgmental or stigmatizing than interactions with a group leader or other parents in a group setting (Budman 2000; Gordon 2000, 2003). One review suggested that adults might learn better online than with face-to-face instruction (U.S. Department of Education 2009), and another review found Internet learning to be similar in effectiveness to traditional methods (Cook et al. 2008). A review by Cook et al. (2010) showed that online and non-computer instruction require similar exposure time, but instructional strategies to enhance feedback and interactivity could increase learning time while enhancing learning outcomes.

While the technology is relatively new, researchers have already successfully adapted proven group-based curricula to an Internet format (Enebrink et al. 2012; Sanders et al. 2012; Taylor et al. 2008) and developed online BPT for parents of children with traumatic brain injury (Wade et al. 2009). Some multi-week Web sessions were supplemented by telephonic or in-person coaching (Sanders et al. 2012; Taylor et al. 2008).

Presentation on the Internet offers the potential for BPT models with alternative instructional designs. Online BPT programs adapted from existing support group curricula tend to be set up as they were originally developed (i.e., organized into a pre-determined sequence of sessions, each building on skills and knowledge from previous sessions). This is called a “tunnel” structure in Internet jargon (Danaher et al. 2005). An alternative BPT approach uses what might be termed a hybrid design, where the learner selects from a menu of problematic parenting scenarios, some of which lead to a tunnel structure (Danaher et al. 2005).

Scenario-based learning, also called problem-based learning, can effectively be integrated into BPT on the Web, and it has been shown to compress training time for adult learners improving judgment and problem-solving skills (Clark et al. 2009). Using a self-paced scenario-driven approach in a BPT context, the parent-user is free to choose a video-based scenario of interest. Many of the same BPT teaching points can be woven into each scenario, and the interested user could have the option of drilling down into related parenting material in the form of text pages or pdfs. Adapted from videodisc (Gordon 2000, 2003) to CD-ROM (Cefai et al. 2010; Segal et al. 2003) and then to the Internet (Feil et al. 2011), a hybrid scenario-based learning design has shown positive BPT effects with multiple populations in different cultures. In a randomized study, Gelatt et al. (2010) developed an online scenario-based intervention tailored to step-parenting family situations. They showed positive effects on parenting and family functioning, but we found no other studies testing non-tunnel designs for online BPT.

Purpose

The research reported here was a randomized trial to test the effectiveness of an online BPT called the Parenting Toolkit. The intervention used a scenario-based hybrid instructional design developed for parents of at-risk adolescents, and it was tested in urban community centers. We hypothesized that the intervention would improve parents’ style of discipline and parent-reported child behaviors, that it would be linked with theoretically relevant psychosocial mediators of behavior change (self-efficacy, behavioral intentions), and that user acceptance would be positive. We also hypothesized that the intervention could be effective when tested remotely, without staff members or the research team being present. Thus, this BPT program evaluation was an effectiveness trial in real-world settings (Flay and Sobel 1983), as opposed to an efficacy trial in a controlled or laboratory setting.

Methods

Intervention Program

The Parenting Toolkit used a video-heavy scenario-based hybrid instructional design adapted from Gordon (2000; Gordon and Stanar 2003). Relevant literature on parenting interventions, parenting focus groups, and consultation with experts guided development of culturally appropriate adolescent-age scripts. A multi-cultural cast of skilled actors was chosen as parent and adolescent video models. The interface design required mouse use but no keyboarding, and on-screen titles and bullets were written for a 4th grade reading level with optional articles written at up to the 8th grade level.

The Parenting Toolkit was designed to increase users’ self-efficacy and behavioral intention to adopt positive parenting behaviors when confronted with oppositional child behavior, which were shown in video vignettes. The problematic parent–child video scenarios included Bedtime, Chores, Curfew, Depression, Grades, Fighting, Friends, Smoking, and Stealing.

As depicted in Fig. 1, users chose a topic (e.g., Stealing) and saw a short “stem” video vignette depicting a problematic situation (Mom discovers adolescent has a new CD player he does not own). After the stem video played, users were asked to choose which of three parental responses was closest to what they would do (lecture the child; enforce rules immediately; ignore it for now). Only one choice was “correct” and led to a positive outcome. Each response-choice linked to a video depicting how that choice might turn out. In the Stealing scenario, the response-choice “lecture the child” showed the parent becoming angry and accusing the boy of dishonoring the family. The adolescent became defiant and stormed out. A 20–60 second video commentary by the parent and then the adolescent followed, reflecting on how the exchange depicted in the vignette made them feel (e.g., the parent was frustrated; she worried he might become a criminal; the unrepentant adolescent felt mistreated and misunderstood).

Fig. 1
figure 1

Abbreviated instructional design of the Parenting Toolkit intervention

Full size image

Following the commentaries, a series of on-screen questions and answers (Q&As) were designed to mimic a parent–therapist counseling session. The Q&As helped users consider how behaviors affected the outcome (“What was wrong with Mom’s approach?”). Each question was answered as a 5–20 second video response by a peer model (“Mom means well, but her emotions got in the way of a conversation that might have helped Roberto understand her message”). Teaching points were shown as text bullets (Stay calm; use a neutral tone; don’t argue). When relevant, “More Details” buttons linked to one-page articles on teaching points (use of consequences; use of neutral language).

The correct video scenario modeled a positive outcome (Mom enforces existing rule with reasonable consequence; she refuses to argue, and she does not lecture). The parent and child commentaries on the correct choice reflect the outcome (Mom was concerned but philosophical that she was handling it correctly; the adolescent grudgingly accepted the consequence and acknowledged that consistent enforcement discouraged theft in the future). The string of Q&As focused on the value of the effective strategies used by the parent.

After the Q&As for a correct response-choice, the user was encouraged to make up a personal Action Plan by selecting up to three parenting skills relevant to that problem situation to practice at home (e.g., stay calm, use “I” statements, make clear rules). Users were encouraged to revisit the Action Plan after finishing each correct response path. From the Action Plan page, users were returned to the home page to select another problem behavior or to browse longer parenting articles on related issues.

Setting and Procedures

The study was conducted in Community Technology Centers (CTCs) in Austin, TX, Oakland, CA, New York City, NY, San Antonio, TX, Edgewood, MD, and Washington, DC. The CTCs had existing computer labs with broadband Internet connections, and four of the six were associated with low-income housing developments.

After the study protocol received approval from the Human Subjects Committee, participants were recruited via local advertising as recommended by each CTC. Emphasis was placed on in-house advertising, including flyers, announcements at meetings, articles in newsletters, and word of mouth, but for each site, we also placed classified ads in the local newspapers.

Potential participants telephoned the research team via a toll-free number. The study was explained, questions were answered, and any individuals still interested were screened for eligibility. Requirements for participation included (a) the parent had primary parenting responsibilities, (b) their child was 11–14 years old, and (c) their child had at least four behaviors from a list of problematic behaviors (e.g., poor grades, trouble at school, drug use, associating with troublesome peers). This list was adapted from the work of Bry and colleagues (Bry et al. 1982, 1988) and had been used in previous community-based research to identify at-risk adolescents (Irvine et al. 1999b). Parents who met eligibility criteria were accepted into the study.

Study Design

The study was a two-arm randomized trial (Fig. 2). Parents screened into the study were mailed a packet containing an informed consent form, a baseline questionnaire (T1), and a stamped return mailer. The participants were asked to read and sign the informed consent, fill out the T1 questionnaire, and return both items in the mailer. When the return packets were received, the parents were randomly assigned to either an immediate treatment (TX) or a control (CT) condition that received no treatment. TX subjects were scheduled by telephone for an appointment to use the program at their local CTC in two visits, 1 week apart. CT subjects were told they would receive program access after the follow-up assessment.

Fig. 2
figure 2

Diagram of study design and assessment strategy

Full size image

One month after TX group members were scheduled for the first visit to the intervention, all subjects were mailed a follow-up questionnaire (T2) with a stamped return mailer. Following the return of the T2 questionnaire, subjects in the CT group were provided access to the program. All subjects were mailed a $40 check for returning the T1 assessment and a $60 check for returning the T2. TX group participants were not given a financial incentive to use the program.

Participants in the TX group reported to their local CTC where the staff assisted them getting started in the program, if necessary (i.e., logging on, use of the mouse), after which the staff-person moved away but remained nearby to provide help if needed. An earphone-headset was provided for each computer to enhance privacy and reduce distraction to nearby computer users. After logging in with a project-assigned password, no keyboarding was required (new pages were accessed by clicking on icons).

The participants were asked to view five of the nine parenting scenarios at visit 1 and the remaining four at visit 2, a week later. Text on the home page suggested that they start at the top of the list of parenting situations and work down. Because this was designed as an effectiveness trial, however, participant visits were not controlled. That is, they could make only one visit, or return twice or more, and they could select parenting situations in whatever order they wished.

Subjects

The 307 participants were randomized, 155 into the TX group and 152 into the CT group (Fig. 2). As shown in Table 1, participants included 277 females (90 %) and 30 males (10 %). They averaged 40.6 years of age (SD = 6.5), and they self-identified as 39.7 % African American, 28.7 % white, 29.0 % Hispanic, 25.7 % other, and 1.3 % Asian. Parenting status included 53.7 % single parents, 33.2 % married, and 12.1 % parenting with a partner. Participants’ children (52.9 % males, 47.1 % females) averaged 13.1 years of age (SD = 1.4). Annual family income was <$10,000 for 21.8 %, $10,000–19,999 for 19.5 %, and $20,000–39,999 for 32.2 % of the sample.

Table 1 Demographics of participants
Full size table

Measures

Psychosocial Constructs

The theoretical framework for program development and for measurement of psychosocial constructs is provided by social cognitive theory (behavioral self-efficacy; Bandura 1977, 1986) and the theory of reasoned action (behavioral intentions; Ajzen and Fishbein 1973, 1979; Fishbein et al. 1991). Both self-efficacy (21 items) and behavioral intentions (12 items) were measured with seven-point scale items developed to the content in the Parenting Toolkit. Self-efficacy item response options ranged from “not at all confident” to “extremely confident,” and intention item response options from “not at all likely” to “extremely likely.” Items were summed to create scale scores, and reliability estimates were 0.90 and 0.83 for self-efficacy and behavioral intentions, respectively.

Discipline Style

Positive changes in parenting approaches to discipline can indicate improved parenting skills. They were assessed with the 13-item Parenting Scale, Adolescent version (PSA; Irvine et al. 1999a), an adaptation of the 30-item Parenting Scale (Arnold et al. 1993) that measures self-reported disciplining “mistakes” by parents of at-risk adolescents. For each item, a parenting mistake was contrasted with a more effective alternative to anchor the opposite ends of a seven-point scale. For instance, a stem “When my child misbehaves,” identifies the topic of the item; the negative anchor is “I raise my voice and yell;” the positive anchor is “I speak to my child calmly.” The PSA is composed of two factors: Overreactivity measures parent coerciveness, and Laxness measures parental follow-through. It was validated for use with adolescents (Irvine et al. 1999a). It has 3-month test–retest reliability of 0.72–0.80 between the two factors and is correlated with the Beck Depression Inventory (Beck et al. 1988), the Child Behavior Check List (Achenbach 1991), and the Parent Daily Report (Chamberlain and Reid 1987). In a randomized trial evaluating the efficacy of parent training with parents of at-risk adolescents, the PSA assessed changes in parenting skills up to 1 year later (Irvine et al. 1999b). In this study, items were summed to create the scores. The Laxness factor had an alpha of 0.74, and the Overreactivity factor had an alpha of 0.71.

Eyberg Child Behavior Inventory (ECBI) Questionnaire

The ECBI is a widely used rating scale that measures parent-reported conduct problems in children, which would be expected to decrease if parenting skills improve. The ECBI assesses the frequency and severity of disruptive behaviors at home and the extent to which parents find the behaviors troublesome. It consists of 36 items listing typical child problem behaviors of children with conduct disorders. Each behavior is rated on a seven-point intensity scale that indicates how often a behavior is perceived to occur and a yes/no scale indicating whether the behavior is currently a problem for the child. The ECBI has been normed across socio-economic levels, and for two decades it has been shown to be sensitive to intervention effects in treatment studies (Colvin et al. 1999). Re-standardized by Colvin et al. (1999), the Intensity and Problem scales correlate strongly (r = 0.75), with internal consistency scores of 0.95 and 0.93, respectively, and 3-week test–retest reliability of 0.86 and 0.88, respectively (Robinson et al. 1980). In this study, items were summed to create the scores, and the internal consistency reliabilities were 0.89 and 0.92 for the Intensity and Problems scales, respectively.

User Satisfaction

TX group participants responded to five items relating to their satisfaction with the Parenting Toolkit. For each question, participants were asked to rate their opinions on a seven-point scale, with responses ranging from “not at all…” to “extremely…” The questions asked about satisfaction, usefulness, ease of use, enjoyability, and willingness to recommend the program to a friend.

Statistical Methods

Preliminary Analysis

Study groups were compared on the demographic characteristics in Table 1 as well as all baseline measures of study outcomes. Groups did not statistically differ on any measure at p <0.05, indicating randomization produced initially similar groups.

Of the 155 TX participants, 92 (59 %) visited a CTC to use the Parenting Toolkit website, but a total of 275 participants (90 %) completed the follow-up assessment. Rates of overall attrition did not differ significantly across experimental condition (TX = 13 %, CT = 7 %; χ 2 [1,307] = 3.27, p = 0.071). Participants who completed both assessments (n = 275) were compared with those who completed only the baseline assessment (n = 32) on study demographic characteristics and baseline measures of the study outcomes. No statistically significant differences at p <0.05 were found.

Missing Data

Rates of missing data were 2 % at T1 and 10–12 % at T2. Multiple imputation was used to replace missing values following best-practice recommendations (Graham 2009). Missing data were imputed using the IVEware software (Raghunathan et al. 2002), which uses all available data to impute missing data via a sequential regression approach. The observed and imputed data were compared to ensure they showed similar distributions (Abayomi et al. 2008). Missing data points were replaced with imputed data in 20 data sets, which were analyzed separately. Model parameters and standard errors, which incorporate within- and between-model parameter variability, were combined using SAS MIANALYZE (SAS Institute Inc. 2011) following methods described by Rubin (1987) and Barnard and Rubin (1999).

Analytic Models

First, an intent-to-treat analysis using the imputed data was conducted using analysis of covariance (ANCOVA) models (Table 2), with baseline measure of the outcome as a covariate and study condition as a two-level predictor (1 = treatment, 0 = control). Models were fit with the PROC GLM procedure in the SAS (SAS Institute Inc. 2011). Effect sizes were estimated by averaging eta-square values across the 20 models resulting from imputed data.

Table 2 Descriptive statistics and results of condition effects from analysis of covariance models with intent-to-treat sample
Full size table

Next, ANCOVA models were used to compare participants compliant with the research protocol (Table 3). That is, TX participants who visited the intervention website and completed the posttest assessment (n = 90) were compared with CT participants who completed the posttest assessment (n = 140). Missing data for this complier subsample were negligible (0 % at pretest and 0 to 2 % at posttest), and thus non-imputed data were used.

Table 3 Descriptive statistics and results of condition effects from analysis of covariance models with compliant sub-sample (those who viewed the Parenting Toolkit and completed the follow-up assessment)
Full size table

Results

Program Use

Users spent an average of 62.9 min (SD 23.7) at visit 1 and 40.5 min (SD 19.3) at visit 2. Contrary to their instructions, 17 individuals visited all nine modules at visit 1 and did not return for visit 2. Average use time for individuals making one visit was 68.3 min (SD 28.3).

Program Effectiveness

Descriptive statistics for the dependent measures and comparisons of the two conditions from the intent-to-treat analysis are summarized in Table 2. The TX group differed significantly from the CT group on three of the six outcome measures, and group means show significant differences in the predicted direction. Effect size estimates indicate significant differences were small in magnitude.

Comparison of the two conditions using only the compliant subsample shows the TX group differed significantly from the CT group on all six outcome measures in the predicted direction (Table 3). Effect size estimates indicate the significant differences ranged from small (ECBI Problem scale, eta-square = 0.017) to moderate (behavioral intentions, eta-square = 0.073; parenting self-efficacy, eta-square = 0.072).

These results indicate modest effects for the intent-to-treat sample and more robust effects for the study-compliant sample. Results of the compliant sample show the discipline style of participants, as measured by the PSA, changed if they viewed the Parenting Toolkit. That is, scores of the TX group indicate that these parents would be less likely to overreact and use harsh responses during disciplinary interactions with their children than would control parents. Similarly, TX group parents’ scores indicted that they would be less lax (more likely to follow through with promised consequences) than would CT group parents. In addition, parents who viewed the Parenting Toolkit reported a reduction in problem behaviors as measured by the ECBI Intensity scale. Lastly, compared to the CT group, TX parents reported greater gains in both self-efficacy and behavioral intentions to engage in positive parenting practices.

User Satisfaction

On our seven-point scale, higher scores indicated users were quite satisfied with the Parenting Toolkit (x = 6.1, SD 1.0) and found it to be useful (x = 6.1, SD 1.1). They found it easy to use (x = 6.3, SD 1.3) and enjoyable (x = 5.6, SD 1.2). The scores for the likelihood that participants would recommend the program to a friend were quite high (x = 6.6, SD 0.14). Time of use was not correlated with any satisfaction scores.

Discussion

The research presented here suggests that relatively brief exposure to online BPT might influence parenting skills 1 month after first exposure. The results on the PSA, a measure of discipline styles, suggest that parent-users would be less likely to overreact during disciplinary interactions, and their improved laxness scores suggest that they would be more consistent with discipline. These results are consistent with decreases in intensity and problem scores on the ECBI, although the problem scores did not reach significance. Results also showed significant improvement on measures of behavioral intentions and self-efficacy related to positive parenting skills. Social cognitive theory and the theory of planned behavior suggest that positive changes in parent self-efficacy and intention should be linked to positive changes in parenting behavior. Posttest results support this supposition for TX group parents at the 30-day follow-up.

In sum, the results suggest that the Parenting Toolkit program achieved measurable learning with high user satisfaction, which meets the federal criteria for successful website development (www.usability.gov). The results also suggest that a validated online BPT might provide parenting skills in a real-world environment without monitoring by experts. This outcome has intriguing implications for the future of BPT for the public on the Internet.

This research represents an independent validation of a non-sequential Internet instructional design pioneered by Gordon (2000, 2003) and extends his research by showing the potential efficacy of this approach for parents using computing centers to access the Internet. Because it is asynchronous learning, the instructional design sometimes resulted in participants being exposed to certain elements (e.g., monitoring, praise) after others (e.g., rules, consequences), which might not be preferred by a traditional group leader or curriculum designer. This approach allows some teaching points (e.g., neutral language, praise, consistent discipline, natural consequences) to be reiterated in different contexts across scenarios. Knowles (1984) postulated that for adults, the learning experience (including mistakes) should be the determinant of the learning activities; adults are more interested in learning subjects that have immediate relevance to their personal life and prefer learning that is problem-centered rather than content-oriented. The scenario-based instructional philosophy, however, presents challenges to the developer to offer authentic real-life scenarios and skill-building elements that will achieve desired learning goals even when the parts that make up the whole are accessed in an uncertain order. Additional research is needed to sort out the influence of these factors, as well as the best BPT approaches for online learning.

While this research supports the potential efficacy of non-sequential scenario-based BPT instructional designs on the Internet, caution is needed. Support group research such as PMTO (Forgatch et al. 2009; Patterson et al. 2010), Triple P (Prinz et al. 2009; Sanders et al. 2012), and Webster-Stratton’s Incredible Years (Taylor et al. 2008; Webster-Stratton and Reid 2010; Webster-Stratton et al. 2011) has produced substantive effect, some of which can be measured 9 to 10 years later (Forgatch et al. 2009; Webster-Stratton et al. 2011). The idea that even a few hours of exposure to video-based online BPT can replicate the effects of 8–12 weeks of support group BPT is appealing, but we should stay skeptical pending further validation. We found no research comparing online tunnel model BPT training, with or without coaching, to the scenario-based learning as reported here, so the comparative efficacy of the two models is unclear.

While scenario-based learning might be appealing to parents focused on a specific parenting issue, they might not find the relevant problem for their needs, and even if they do, the BPT might not offer the requisite skills to effectively address the problem. More importantly, the skills might not generalize to other parenting issues. Scenario-based learning could be a hook to improve user engagement, but the relationship of user engagement to behavioral effects has so far received little attention in the literature (Davies et al. 2012).

Some reports suggest that over time, online BPT might be more cost-effective per person served than group BPT (Enebrink et al. 2012; Furlong et al. 2012; Gordon 2003). A cost–benefit analysis of an online BPT program compared to implementation of a support-group type program including therapist training will be a complex but interesting challenge for future researchers. Development costs, especially the expense required to produce video-heavy web programs, have received little attention (Metzler et al. 2012; Sanders et al. 2012). To develop the Parenting Toolkit, we used the best production company our budget could afford with the rationale that our actors and scenarios must be believable if they are to be credible to parents used to watching dramas on TV. That is not to say that spending less will necessarily decrease the effect of the modeling video. Webster-Stratton culled through thousands of hours of videotapes of parent–child interactions in her laboratory to find segments suitable for teaching points in Incredible Years, but few researchers or clinicians have access to that type of resource. We found no research into the most efficacious methods to produce behavioral modeling video, much less Internet development approaches to produce substantive results.

Finally, stand-alone online BPT programs should be considered carefully because they could potentially create a risk for participants by over-simplifying solutions to complex psychosocial personal or family problems best addressed with the assistance of therapists (Enebrink et al. 2012; Taylor et al. 2008). This suggests a need to develop programmatic contingencies to assess risks and urge some users to seek professional help. Online BPT programs might address this risk if they have built-in coaching by phone or email, but assessing the risk of fully automated Internet BPT programs and developing solutions are issues still to be addressed.

Limitations

This research had several limitations. The results reported here are based on user self-report with only a 30-day follow-up; the PSA and the ECBI were the only validated assessment instruments; and most of the parents were from low-income minority populations in urban environments. Our results would inspire greater confidence if they were substantiated by in vivo observations of parent–child interactions over at least a year within a more diverse sample of study participants. Even then, the clinical significance of the results, which could differ from statistical significance, would need to be established over years, as it has been with support-group BPT research (Forgatch et al. 2009; Webster-Stratton et al. 2011). So far, other scenario-based BPT studies have shown effects over a more modest time frame. The Parenting Wisely CD-ROM program showed effects for up to 6 months (Cefai et al. 2010), and Gelatt et al. (2010), using a scenario-based Internet design, showed effects on stepparents at 16 weeks.

Additionally, the relatively small number of TX participants to use Parenting Toolkit is troubling. One explanation might be that the requirement for TX participants to visit a CTC to view the intervention was greater than the effort required of CT participants who only filled out surveys. Better TX participation might have occurred had the intervention been accessible from homes. In future real-world online BPT applications, we believe that participation will be enhanced with reminders calls, emails, or incentives for participation.

Conclusions

We believe the results presented here support further research into scenario-based Internet BPT. The Web offers the potential for broad dissemination to parents who might not attend an in-person session because of logistics or other factors. Internet BPT might be offered in communities to promote prevention or early identification of problematic child behavior. With online BPT available to all, agencies could prioritize services for parents who might benefit from the more expensive counseling and therapist-led support group experience. Even parents who are just casually interested in developing or improving their parenting skills could benefit from having validated BPT available.

Clearly, much remains to be learned. The relative efficacy of scenario-based versus traditional tunnel curriculum designs and the potential place of supplemental in-person or telephone coaching offer challenges for future researchers. Woven into the picture should be the determination of how to best implement research-validated BPT in non-research settings.