Adolescence, or the period from ages 10 to 19, is a time of considerable sexual development and adoption of sexual risk behaviors (Kar et al. 2015; Sawyer et al. 2018). Adolescent girls in the United States are at unique risk of negative sexual health outcomes. Up to 25% of sexually active adolescent girls have a sexually transmitted infection (STI; Forhan et al. 2009), which can lead to complications, such as ectopic pregnancy, pelvic inflammatory disease, infertility, and cervical cancer (Centers for Disease Control and Prevention [CDC] 2018). Adolescent girls are also at risk of unintended pregnancy, with 194,377 babies born to adolescents in 2017 (Martin et al. 2018). Beyond these individual consequences, STIs and unplanned pregnancy confer significant costs to the U.S. healthcare system: over $36 billion a year (CDC 2013; Sonfield and Kost 2015). With rates of STIs and unintended pregnancy among the highest in the developed world (The Henry J. Kaiser Family Foundation 2014), the U.S. must prioritize sexual health, particularly among adolescent girls.

Adolescent girls with emotional and behavioral difficulties (EBDs) are at a heightened risk of negative sexual health outcomes (Connell et al. 2009). EBDs refer to a range of experiences including mental health problems (e.g., depression, anxiety) and problem behaviors (e.g., substance use, delinquency; Connell et al. 2009; Garner et al. 2013). EBDs are linked to several sexual risk behaviors that are associated with STIs/HIV and unplanned pregnancy, including earlier sexual debut, multiple sexual partners, and inconsistent condom use (Aalsma et al. 2010; Cavazos-Rehg et al. 2010; Donenberg et al. 2001; Schuster et al. 2013). Yet, no currently available evidence-based sexual health interventions specifically target youth with EBDs (Office of Adolescent Health [OAH] 2016), and only three target other high-risk youth (i.e., youth in alternative schools, homeless youth, and incarcerated youth; OAH 2016). There is a need to establish efficacious sexual health interventions to serve adolescent girls with EBDs.

Recently, a brief, web-based intervention called HEART (Health Education and Relationship Training) was developed to enhance assertive communication skills and improve sexual health outcomes for adolescent girls (Widman et al. 2016). The program is based on the Reasoned Action Model (Fishbein and Ajzen 2010) and emphasizes the importance of sexual communication in promoting condom and contraceptive use (Schmid et al. 2015; Widman et al. 2014). A school-based sample of 222 adolescent girls found HEART enjoyable and engaging (Widman et al. 2017). Participants showed improvement in communication intentions, sexual assertiveness skills, STI/HIV knowledge, condom self-efficacy, and safer sex attitudes and norms (Widman et al. 2018). These studies support the acceptability and efficacy of HEART among a school-based sample; however, the programs’ applicability to adolescent girls with EBDs has not been established.

There are reasons to expect HEART can improve sexual health outcomes among adolescent girls with EBDs. First, the program is brief, lasting 45 min, and is completed in one sitting, meaning intense resources and a lengthy time commitment are not required for completion. Second, it is web-based. A growing literature supports the utility of digital interventions for adolescents (Lenhart et al. 2015), and high-risk youth, such as adolescent girls with EBDs, may be more engaged with their health when they can access health information online (Gaskin et al. 2012). Third, and finally, HEART focuses on building communication skills that are vital for negotiating condom use, especially among adolescent girls with EBDs who tend to have communication skill deficits, such as verbal negotiation difficulties (Cross 2011).

In addition to HEART’s potential to improve sexual health outcomes for adolescent girls with EBDs, evaluating HEART in this population contributes to the field of prevention science, which seeks to identify best practices for ameliorating health concerns (Sloboda and Petra 2014). Specifically, we contribute to the area of prevention science focused on translating efficacious interventions into practice (Sloboda and Petra 2014). One evaluation in a single population and/or setting is insufficient for understanding the transferability of the intervention, that is, whether efficacy outcomes from an intervention in one context will be found in a new population or context (Schloemer and Schroder-Back 2018). Given the impact of one’s group identities on health, an intervention may not be equally appealing, appropriate, or efficacious among different groups (Barrera et al. 2013). In the case of our study, adolescent girls with EBDs may not respond the same to HEART as adolescent girls from a general sample. As the array of efficacious sexual health interventions expand, we must ensure we are evaluating the feasibility, acceptability, and efficacy and effectiveness of these programs among varied populations and in real-world settings (Bull 2018; Valentine et al. 2011).

To these ends, the purpose of this study was to understand whether HEART can be implemented with efficacy among a sample of adolescent girls with EBDs in community-based settings through completion of four aims: (1) to determine the feasibility of recruiting adolescent girls with EBDs for a web-based intervention through examination of the success of our recruitment procedures, (2) to assess the feasibility of administering the program in community-based settings through examination of program fidelity, (3) to examine the acceptability of HEART using a self-report measure, and (4) to report preliminary evidence of the efficacy of HEART among a pilot sample of adolescent girls with EBDs using a pre-post evaluation within the intervention group and compared to a non-equivalent comparison group matched to the intervention group by propensity score analysis.

Method

Participants

Because this study was a preliminary evaluation of HEART among adolescent girls with EBDs, we used a non-equivalent comparison group design. Our intervention group was recruited specifically for this study, whereas our comparison group consisted of adolescent girls who completed a program called Growing Minds (Burnette et al. 2017) and were matched to the intervention group through propensity score analysis. Additional details about the intervention and comparison groups appear below.

Intervention group

Between February 2017 and March 2018, 52 participants in the southeastern U.S. were recruited from seven community-based organizations that provide services to high-risk youth who demonstrate several needs including: homelessness, court involvement, and behavioral or emotional problems. Several methods were used to recruit participants, including referrals from organization staff, face-to-face contact with parents or adolescents, requests for participants on email lists, and recruitment flyers in organizations. Adolescent girls aged 12–19 who were fluent in English were eligible for the study. Because of the early age of initiation and increased sexual risk behavior among adolescent girls with EBDs, we used a wide age range in our recruitment. Twelve was deemed the minimum age because our community partners believed HEART may benefit 12-year-old adolescent girls within their organizations. The presence of EBDs was used as a selection criterion for analyses but not for participation in the study; thus, five participants completed the intervention and were later excluded from analyses because they did not meet pre-specified criteria for having EBDs (see measure section below).

Comparison group

Our comparison group was 115 adolescent girls who completed an attention-matched web-program called Growing Minds (Burnette et al. 2017), focused on cultivating academic growth mindsets, in Fall 2015. All 10th grade girls (n= 371) from four southeastern U.S. high schools were recruited using active consent during the initial evaluation of HEART (Widman et al. 2018). Of the 78% of students who returned parent consent forms, 79% of parents granted consent. Presence of EBDs was not an inclusion criterion for analyses among the comparison group.

Procedure

Intervention group

Participants who were under 18 years old returned a parent/guardian consent form and completed a participant assent form prior to data collection. Participants who were 18 or older completed a consent form prior to data collection. After consent/assent procedures, participants completed pretest measures via an online survey. Then, they completed HEART via laptop computer in a private room, either in the organization from which the participant was recruited or our university laboratory space, based on the participant’s preference. Upon program completion, participants completed a two-minute audio task measuring sexual assertiveness skills. Lastly, participants completed an online posttest survey. Participants were compensated $20 in gift cards. This study was approved by the University Institutional Review Board.

Comparison group

After consent/assent procedures, participants completed a computerized pretest survey during school time. Next, they were randomly assigned to HEART or Growing Minds. In total, 222 participants were randomized, 115 to Growing Minds. In the following weeks, participants completed the intervention, sexual assertiveness skills assessment, and immediate posttest survey individually in private rooms. Participants received $50 in gift cards for their participation. This study was approved by the University Institutional Review Board.

Programs

HEART

A full description of HEART is available elsewhere (Widman et al. 2016). Briefly, HEART contains five modules that target (1) safer sex motivation, (2) STI/HIV knowledge, (3) sexual norms/attitudes, (4) sexual self-efficacy, and (5) sexual communication. All modules incorporate features such as information, videos, games, and skills training with self-feedback. Assertive communication is emphasized throughout the program, but the sexual communication skills module focuses specifically on skill development, using a role-playing exercise. HEART prompts users to consider their sexual values, make healthy and informed sexual choices, and implement these choices, making it appropriate for adolescent girls across development and with differing sexual histories.

Growing minds

The Growing Minds program focuses on cultivating academic and social growth mindsets (for a full program description, see Burnette et al. 2017). Growing Minds includes five modules that are comparable in format and materials to HEART. This program is compatible with previous mindset work but provides two unique components: (1) teaching about growth mindsets and (2) incorporating role models to reinforce growth mindsets. Both HEART and Growing Minds require active participation and include similarly engaging content, such as videos and quizzes. Further, both programs take approximately 45 min time to complete.

Measures

Participant characteristics

Sociodemographic (e.g., age, race/ethnicity, sexual orientation, and parent education) and romantic/sexual characteristics (e.g., relationship status, sexual activity, and condom use), consistent with the original assessment of HEART, were assessed. These measures were administered at pretest only and were examined as potential matching variables for the propensity score analysis.

Emotional and behavioral difficulties

Intervention participants had to meet criteria for at least one of the EBDs described below to be included in study analyses. These measures were included at pretest only.

Delinquency

Thirteen items from the delinquency behavior scale of The Rochester Youth Development Study assessed antisocial and delinquent behavior (Thornberry et al. 2003). Participants responded to items such as, “In the past 30 days, have you tried to buy or sell things that are stolen” with either “Yes” = 1 or “No” = 0. Participants who had engaged in at least one delinquent behavior in the past 30 days (i.e., scored 1 or more) were eligible for inclusion in analyses.

Substance use

Three items assessed how many days in the past three months participants had used alcohol, marijuana, or other illegal drugs (e.g., opioids) (Kano et al. 2016), with options ranging from “0” to “9 days or more”. Adolescents who had engaged in any substance use in the past three months were eligible for inclusion in analyses.

Mental health symptoms

The Strengths and Difficulties questionnaire (Goodman et al. 1998) was used to assess the presence of emotional or behavioral mental health symptoms. The questionnaire has four subscales: emotional symptoms, conduct problems, hyperactivity/inattention, and peer relationship problems. Participants responded on a scale from “Not True” to “Certainly True” for a maximum score of ten on each scale. Scores on this subscale are categorized as “Normal,” “Borderline,” or “Abnormal” based on normed data (Goodman et al. 1998), with 20% of the population being classified as “Borderline” or “Abnormal”. Participants in this study who scored “Borderline” or above on any of the subscales (cut-off scores: Emotional Problems = 6; Conduct Problems = 4; Hyperactivity/Inattention = 6; Peer Problems = 4) were eligible for inclusion in analyses.

Outcome Measures

Feasibility

To assess feasibility of recruitment, we used several measurements: (1) number of participants recruited, (2) time to recruit participants, (3) number of sites contacted, (4) number of sites that agreed to aid in the recruitment process, (5) number of sites that referred participants, and (6) number of participants recruited that met inclusion criteria. To assess the feasibility of program administration, we examined: (1) time to complete intervention, (2) number of participants who received the complete intervention, (3) number of participants who experienced technological or other problems during program completion, and (4) additional barriers experienced to program administration unrelated to program errors.

Acceptability

Participants responded to questions, adapted from previous measures (Bauermeister et al. 2015; Jemmott et al. 2005; Paiva et al. 2014), that assessed six aspects of acceptability, including, (1) how much they liked the program, (2) how much they learned from the program, (3) how much the program kept their attention, (4) intent to return to the website again, (5) whether they would recommend the website to a friend, and (6) whether they would use the information from the program in the future. The first three questions were scored a four-point scale (1 = “Not at all,” 4 = “A lot”), whereas the last three questions were dichotomously scored (0 = “No/Not Sure,” 1 = “Yes”). These measures were administered at posttest only.

Program efficacy

Our outcome evaluation measures assessed diverse aspects of sexual health shown to be important to sexual decision making in health behavior theory (Fishbein and Ajzen 2010), including measures of communication intentions, sexual assertiveness skills, condom use intentions, STI/HIV knowledge, sexual self-efficacy, condom attitudes, and condom norms. All measures, except sexual assertiveness skills, were administered at pretest and posttest to assess improvement in sexual health knowledge, attitudes, and behavioral intentions after completing HEART.

Communication intentions

Three items, adapted from the AIDS Risk Behavior Survey (ARBA) (Donenberg et al. 2001) assessed the likelihood of communicating with a dating partner about sexual topics, including: (1) sexual limits/boundaries, (2) STIs/pregnancy, and (3) condom use. Participants indicated 0 to 100% likelihood of communicating about each topic with a dating partner in the next three months. Scores were averaged to create a composite score, with higher scores indicating greater likelihood of sexual communication (Intervention: α = 0.83; Comparison: α = 0.83).

Behavioral sexual assertiveness skills assessment

Participants’ skills at refusing unwanted sexual activity and negotiating condom use were assessed with an audio recorded role-play task (adapted from Forsyth et al. 2016). Participants listened and responded out-loud to three role-play situations involving: (1) pressured intercourse from a steady romantic partner, (2) pressured intercourse from a casual partner, and (3) pressured unprotected intercourse from a partner. Responses were audio recorded, transcribed by trained research assistants, and rated based on three facets from an established coding system (Forsyth et al. 2016): (1) refusal of behavior, (2) giving a reason for the refusal, and (3) verbal assertiveness (range for each code: 0 = absent, 1 = moderate, 2 = high). Four coders blinded to study condition rated comparison group responses (interrater agreement = 84%), and two of the same coders rated intervention group responses (interrater agreement = 79%). Scores were averaged to obtain an index of sexual assertiveness skills for all participants who had at least one valid response in which they followed task instructions. Participants who did not follow instructions were excluded (n= 22).

Condom intentions

One item from the ARBA (Donenberg et al. 2001) was used to assess the degree to which participants planned to use condoms. Participants selected a response from 0 to 100%, indicating likelihood of condom use in the next three months if they had sex, with higher scores indicating condom use in a greater percentage of sexual encounters.

STI/HIV knowledge

Twelve items assessed participants’ knowledge of STIs/HIV (e.g., “Most people know when they have an STI”). The number of correct responses was calculated, with higher scores indicating greater STIs/HIV knowledge (possible range = 0–12). This assessment was adapted from previous sexual health questionnaires (Brown et al. 1992; Morton et al. 1996).

Sexual self-efficacy

The 7-item self-efficacy for HIV Prevention scale (Brown et al. 2014) was used to measure confidence in using HIV preventative behaviors. Example items included: “How sure are you that you could talk to your partner about safer sex?” and “How sure are you that you could have condoms available when you need them?” Responses were on a 4-point scale from 1 (Couldn’t do it) to 4 (Very sure). Scores were averaged with higher scores representing greater confidence in HIV prevention skills (Intervention: α = 0.73; Comparison: α = 0.70).

Condom attitudes

Condom use beliefs (e.g., “Condoms create a sense of safety”) were measured using the Safety subscale and Effect on Sexual Experiences subscale of the Condom Attitudes Scale – Adolescent Version (St. Lawrence et al. 1994). Participants responded on a scale from 1 (Strongly Disagree) to 5 (Strongly Agree). Scores were averaged to create a composite score with higher scores indicating more positive condom attitudes (Intervention: α = 0.51; Comparison: α = 0.52).

Condom norms

Participants responded to three items (Basen-Engquist et al. 1999) on a scale from 1 (Strongly Disagree) to 5 (Strongly Agree) to assess perceptions of their peers’ opinions on condoms (e.g., “Most teenagers believe condoms should always be used if a person my age has sex”). Scores were averaged to create a composite score with higher scores indicating views that peers’ condom attitudes are positive (Intervention: α = 0.86; Comparison: α = 0.91).

Data Analysis

To account for the non-equivalent comparison group design, we used propensity score analysis (PSA), which matches intervention and comparison group participants to mimic a randomized controlled design. PSA requires a complete dataset without missing data. Therefore, prior to PSA, we performed multiple imputation on the combined dataset. Data were analyzed with the Missing Values package in SPSS Version 24. Approximately 0.60% of data points were missing from 54 participants on 11 pretest measures (parent education, sexual orientation, sexual assertiveness, delinquency, mental/behavioral health, communication intentions, condom intentions, STI/HIV knowledge, sexual self-efficacy, condom attitudes, and condom norms) and six posttest measures (communication intentions, condom intentions, STI/HIV knowledge, sexual self-efficacy, condom attitudes, and condom norms). No variable had more than 8% missing data points. Although Little’s chi-square test (Little 1988) suggested that the data was Missing Completely at Random (MCAR) for both the comparison group, (χ2[3401] = 3345.83, p = 0.75) and the intervention group (χ2[195] = 0.00, p = 0.99), in order to retain the full sample, we used multiple imputation on the combined dataset for N= 10 imputations, consistent with established guidelines (Graham et al. 2007). All data values were imputed at the item level, were constrained by available values within the dataset, and were set to predict imputed values and be imputed, except for the posttest items that were imputed only. Sexual assertiveness skills data was not imputed because it was missing not at random.

Second, PSA was conducted using the MatchIt package (Ho et al. 2011) in R version 3.4.3 to match intervention and comparison group participants and mimic a randomized, experimental design. Based on established guidelines (Brookhart et al. 2006), we included demographic and pretest assessments of outcome variables for matching that were significantly related to group membership (intervention vs. comparison) at a p< 0.15 level, which included age, White race (0 = “Non-White”, 1 = “White”), Black race (0 = “Non- Black”, 1 = “Black”), parent education (0 = “high school degree or less”, 1 = “some college or more”), sexual activity status (0 = “not sexually active”, 1 = “sexually active”), and pretest scores on sexual assertiveness self-report, communication intentions, condom intentions, condom attitudes, sexual self-efficacy, and STI/HIV knowledge. Although there was no statistical difference between intervention and comparison group on condom norms, we also included this variable in the PSA because of its association with posttest assessments. Following PSA, balance diagnostics showed an improvement in balance across 7 of the 11 variables significant prior to matching (Rubin and Thomas 2000), with an overall balance improvement of 95% (see Table 1).

Table 1 Standardized mean differences and sample descriptive statistics of pretest measures
Full size table

Third, we addressed the four aims of the study. To assess feasibility of recruitment and program administration and acceptability (Aims 1–3), we examined descriptive data based on the aforementioned measurements. Then, to assess preliminary efficacy (Aim 4), first, we examined unadjusted within-intervention differences on outcome variables from pretest to posttest using paired sample t-tests. Then, we conducted weighted linear regression analyses. Although controlling for all covariates used in PSA is recommended to reduce possible remaining bias (Rubin and Thomas 2000; Schafer and Kang 2008), with the current sample size, we would have had inadequate power to detect differences between the intervention and comparison groups. Instead, for each analysis, we included the pretest measurement of each variable as a covariate, except for the analysis of behavioral sexual assertiveness skills, which were only measured at posttest. Further, age was controlled in regression analyses to account for the significant difference in age between intervention and comparison group following PSA (p = 0.03).

Results

Feasibility of Recruitment

Initially, we contacted one organization that serves high-risk adolescents who agreed to aid in study recruitment and planned to recruit 50 participants from this organization over 1 year. After seven months, 16 participants had been recruited. When we recognized we could not feasibly recruit using this method, we asked additional sites to aid in study recruitment. Twelve additional sites were contacted and agreed to aid in recruitment; however, participants were only recruited from seven of these sites because adolescents in the remaining five did not express interest in the study, did not return parent consent forms, or were unlikely to meet inclusion criteria. After 1 year, 40 participants had been recruited, and 12 participants were recruited over the next five weeks for a total of 52 participants over an approximately 13-month period. Ninety percent of participants (n = 47) met criteria for having emotional or behavioral difficulties.

Feasibility of Program Administration

On average, participants completed the program in 44 min; however, four participants experienced technological problems that necessitated restarting a module, and their completion times ranged from 49–76 min. One participant took meaningfully longer (95 min) despite no technological problems, likely due to inattention or slower processing speed. Overall, 94% of participants received the full intervention dose as intended. However, due to a temporary glitch in the program, two participants received only four of the five modules. Further, one participant was given an incorrect login identification and did not complete the intervention in the intended order. No other technical difficulties were reported.

Acceptability

The majority of HEART participants liked the program: “some” (19%) or “a lot” (77%; M= 3.70; SD = 0.62), learned “some” (17%) or “a lot” (77%; M = 3.66; SD = 0.73), and felt the program held their attention “some” (38%) or “a lot” (57%; M = 3.54; SD = 0.62), with only one or two participants indicating “not at all” on these items. Most said they would return to the website (77%), recommend the program to a friend (91%), and would use what they learned in the future (98%).

Program Efficacy

Comparing pretest and posttest outcome scores among HEART participants, improvements were seen on all seven variables (see Table 2), communication intentions [t(46) = −4.52, p< 0.001, d= 0.66], condom intentions [t(46) = −3.21, p= 0.001, d= 0.47], condom attitudes [t(46) = −5.25, p < 0.001, d= 0.77], condom norms [t(46) = −3.06, p = 0.002, d= 0.044], sexual self-efficacy [t(46) = −4.95, p < 0.001, d= 0.71), and STI/HIV knowledge [t(46) = −9.91, p< 0.001, d = 1.44].

Table 2 Preliminary efficacy of HEART: differences from pretest to posttest among intervention group only (N = 47)
Full size table

Examining differences between HEART participants and Growing Minds participants, HEART participants had significantly better scores compared to Growing Minds participants on six of the seven outcomes (see Table 3), including communication intentions (B = 10.42, p= 0.003, d= 0.38), sexual assertiveness skills (B = 0.21, p < 0.001, d = 1.13), STI/HIV knowledge (B= 3.75, p < 0.001, d= 1.65), sexual self-efficacy (B = 0.44, p < 0.001, d = 0.87), condom attitudes (B = 0.44, p < 0.001, d = 0.83), and condoms norms (B = 0.47, p < 0.001, d= 0.53). However, intervention and comparison group participants did not differ on condom intention scores (B = 2.61, p= 0.56, d= 0.10).

Table 3 Preliminary efficacy of HEART: comparison of intervention group to propensity-score matched control
Full size table

Discussion

Adolescent girls with EBDs are at an increased risk of negative sexual outcomes, but few sexual health interventions have been evaluated among this population to reduce this risk. The current study addressed this gap in the current sexual health intervention literature by investigating whether a web-based sexual health intervention called HEART could be implemented with efficacy among a unique sample of adolescent girls with EBDs from community-based organizations in the southeastern U.S. Our results suggest (1) HEART can be feasibly implemented among adolescent girls with EBDs, (2) HEART is highly acceptable among adolescent girls with EBDs, and (3) HEART may improve sexual health outcomes for adolescent girls with EBDs. These results are important given the urgent need to improve sexual health outcomes among adolescent girls with EBDs who are at an elevated risk of STIs and unintended pregnancy (Connell et al. 2009).

First, we discuss the feasibility of recruiting adolescent girls with EBDs to participate in HEART. Our initial plan to collaborate with one organization to recruit 50 participants over a 1-year period was not feasible. However, once we collaborated with additional organizations, we reached 52 participants within 13 months. Further, 90% of participants recruited met criteria for inclusion, suggesting that we recruited from appropriate organizations and did not employ overly restrictive inclusion criteria. However, we must note that establishing connections with organizations that were capable and/or willing to aid in the recruitment was a difficult process. In total, we contacted 31 organizations. Of these, 3 declined, 20 did not respond or follow up with inquiries about recruitment of participants, and 12 were willing to aid in recruitment. One organization declined because they were concerned about parents’ reactions to the delivery of a sexual health intervention to their children, whereas the other two organizations declined because partnering with researchers to conduct an intervention study was inconsistent with organization policies. Although we do not know why some organizations did not respond, it is likely that many did not have the time or resources to respond to our inquiry or aid in the recruitment process. Further, lack of interest and engagement and barriers to parental consent resulted in only seven of the willing agencies aiding in the recruitment process. Therefore, recruitment of adolescent girls with EBDs from community-based settings is feasible but dependent upon the availability of organizations in the community that have the time and resources to participate and the ability of researchers to engage in networking efforts to establish organization connections. Researchers and interventionists may have more success recruiting participants in the future if parental permission is waived, as can be done for studies involving minors in which requiring parental consent may unintentionally impede adolescents in need of intervention from participating (Liu et al. 2017). If parental consent is waived, it could reduce strain on organization staff who spent substantial time during our recruitment process informing adolescents and encouraging them to return consent forms. Alternatively, if parental consent cannot be waived, researchers could work with organizations to include consent forms for studies into required organizational paperwork for parents upon adolescents’ entry into organization services.

Second, we discuss the feasibility of administering HEART to adolescent girls with EBDs in community-based settings. Participants completed the program in 44 min on average, consistent with the intended length of the program. All participants who preferred to complete the intervention during their regularly scheduled time in their community-based organization were able to do so. Although technological problems were experienced by a few participants, only two of these problems resulted in the participant missing a module. Our team is continuing to optimize this program to avoid technological problems to the greatest extent possible.

Third, we discuss the acceptability of HEART among adolescent girls with EBDs. An overwhelming majority of participants liked the program, thought it was helpful, were engaged with it, would recommend it to a friend, and would use the information in the future. Further, most participants reported they would return to the website. Notably, these acceptability ratings are comparable to those from participants in the initial trial of HEART (Initial evaluation: 77–95% v. Current study: 79–98%; Widman et al. 2018), and are similar or somewhat better than other evidence-based online interventions for youth (for examples, see Roberto et al. 2007; Shegog et al. 2014).

Lastly, we provide preliminary evidence of HEART’s efficacy in improving sexual health outcomes for adolescent girls with EBDs in community-based settings. Among participants enrolled in HEART, significant improvement was observed on all seven outcome variables. Further, adolescent girls who received HEART improved significantly in their communication intentions, communication skills, STI/HIV knowledge, sexual self-efficacy, condom attitudes, and condom norms compared to the comparison group. Notably, the effect sizes in this study were comparable to those in the original evaluation of HEART (Initial study: ds = 0.03–4.78 vs. Current study: ds = 0.10–1.65) and also with more time-intensive in-person interventions (Johnson et al. 2011). Although we cannot confirm HEART’s efficacy among adolescent girls with EBDs given our nonrandomized matched comparison design, effect sizes from this study will aid us in determining the sample size for a future randomized controlled trial of HEART among adolescent girls with EBDs (Thabane et al. 2010).

Together, our results provide preliminary support for the transferability of HEART to adolescent girls with EBDs to community-based settings. These findings are promising given the noted lack of sexual health interventions available that have specifically targeted or been evaluated among this population. Further, this study is valuable in its focus on understanding whether a sexual health intervention, efficacious among a general sample of adolescent girls, can be implemented with efficacy in a new population. This type of research is largely absent from the literature but is necessary for understanding best practices or health education in the real-world (Brownson et al. 2018) and contributes to the field of prevention science. Nevertheless, we must note some limitations.

Limitations

First, we used a matched comparison design without randomization. Although we used propensity score analysis to approximate a randomized trial, not all differences between the intervention and comparison group could be eliminated on measured variables; however, we controlled for age and pretest measures in primary analyses. Further, we could not account for cohort differences, such as year of data collection or geographical location. Thus, our findings should be treated as preliminary support for the efficacy of HEART among adolescent girls with EBDs. Second, intervention participants completed the pretest assessment, intervention, and posttest assessment in the same day without long-term follow-up assessments, which removed the possibility of participant attrition but also prohibited examining program effects for adolescent girls with EBDs in the long-term. Our past work suggests that HEART’s efficacy in improving HIV/STI knowledge, condom attitudes, and condom self-efficacy is maintained up to 4-months post-intervention, but additional modifications and/or booster sessions may be necessary to increase the potency of the intervention (Widman et al. 2018). Such changes could be made prior to future evaluations of HEART among adolescent girls with EBDs. Third, this sample was recruited from the southeastern United States, and the results may not be generalizable to other parts of the country. A wide age range of adolescent girls were recruited for this study, but there is limited understanding of HEART’s efficacy across adolescence; future research could investigate this question. Further, although unintended, this sample only included racial/ethnic minority adolescent girls. The reason for this demographic composition cannot be definitively determined, but it is possibly a result of systemic bias against adolescent girls of color in the United States. Historically, adolescent girls of color have been criminalized and considered to be of higher risk than White adolescent girls (Chesney-Lind and Eliason 2006), which could have contributed to their involvement in community-based services or their likelihood of being referred to this study. However, given the sexual health disparities among adolescent girls of color, it is important that these adolescent girls were reached by the intervention. Fourth, there were issues with the assessment measures in this study, including several scales with lower internal consistency than desired, particularly the condom attitudes scale; however, the measure was included in the outcome assessment of this study to keep the analyses as consistent with the original evaluation as possible.

Based on these limitations, we suggest that, in the future, HEART is evaluated among adolescent girls with EBDs in a randomized controlled trial with long-term follow-up to better understand the efficacy of this program among this group. We also advocate for continued explorations of the transferability of HEART to new populations; currently, HEART is undergoing evaluation among adolescent boys and girls in Zambia. Further, evaluations of this program, and generally throughout the intervention literature, must go beyond efficacy trials and evaluate feasibility of program implementation and program effectiveness in real-world settings. With an abundance of “evidence-based programs” available and an urgency with which they need to be administered in new settings, we must be able to provide thorough, easily digestible evidence to stakeholders and program decision-makers regarding with whom and in what settings health education intervention will be most successful (Brownson et al. 2018); otherwise, we risk disseminating interventions to settings in which they are inappropriate or even lead to harmful outcomes (Brownson et al. 2018).

To truly improve public health, we must expand our evaluation techniques. By examining the feasibility, acceptability, and preliminary efficacy of HEART in a new population of adolescent girls with EBDs, we not only make strides toward improving the health of these adolescent girls, but we also contribute to the future of prevention science.