Despite advances in the identification of evidence-based interventions that prevent crime, substance abuse, and adverse mental health outcomes, little research has been conducted to guide the effective implementation of those interventions in community practice (Institute of Medicine 2009). One of the most significant challenges for community implementation of preventive interventions is participant retention (Brown and Liao 1992; Ingoldsby 2010).

Since 1997, the Nurse-Family Partnership (NFP), a program of prenatal and infancy home visiting by nurses for low-income mothers bearing first children, has been implemented in 44 states, with over 350 local NFP sites, in over 550 counties, serving over 200,000 families in the USA (Nurse-Family Partnership n.d.). International expansion is being conducted in six other high-income countries. The NFP is designed to improve the health and development of low-income young women and their children by visiting families in their homes from as early in pregnancy as possible through child age 2. The program accomplishes these goals by improving prenatal health behaviors, parents’ care of their children, and parental health and life course (Olds 2002). Since 1977, it has been tested in a series of randomized controlled trials with different populations and shown considerable promise as a means of improving maternal and child health (Olds et al. 2002; Olds et al. 2004; Kitzman et al. 2010; Olds et al. 2010). It has served as the primary source of evidence in support of the Maternal, Infant, and Early Childhood Home Visitation program funded under the Affordable Care Act (Haskins et al. 2009).

As the NFP is replicated in new community settings, every site enters data into a web-based information system that is used to monitor and improve its implementation. From these data, we learned that participant retention and completed home visits in community settings, compared to implementation in the original trials, are lower. Sixty-two percent of the participants in the Denver trial of the NFP completed the program (Korfmacher et al. 1999), for example, compared to 35 % of families in community replication, and NFP nurses in community replication are completing 78 % of the visits completed in the original RCTs (O’Brien et al. 2012).

In order to identify likely causes of lower retention in community practice, we conducted a mixed method study of retention and completed home visits in NFP implementation sites. We found that higher risk families dropped out more frequently than their lower risk counterparts and that even after adjusting for participant risks, there were highly significant differences among sites and nurses in participant retention and completed home visits. A qualitative analysis indicated that successful sites employed nurses who adapted the content and dose of the program to the needs of the families they served (O’Brien et al. 2012). While the NFP is structured, it is designed to be adapted to families’ needs. Nurses who adapted the program more extensively to families’ needs had greater success in retaining and engaging them than did nurses who followed the prescribed dosage and content with less attention to participant needs (O’Brien et al. 2012).

The results of this mixed method study led to the development of a retention intervention (RI), grounded in motivational interviewing (MI) (Miller et al. 2008), in which NFP nurses were supported in giving even greater control over the content and dosage of the program to the families they visited. In a 15-site quasi-experimental study, we found that those sites that received the RI had greater improvements in participant retention and completed home visits from baseline to intervention periods than did matched control sites (Ingoldsby, et al. 2013). Although intervention and control groups were remarkably equivalent on baseline attrition and other characteristics, we were concerned that our selection of RI sites on the basis of their commitment to helping us develop it may have led to the selection of especially motivated sites, which may have accounted for the improvements in retention and completed home visits over time rather than the intervention itself.

We decided to test this retention intervention in a randomized controlled trial before pressing it into practice. In order to increase the RI’s potential for nationwide expansion, we adapted it so that it might be taught to new nurses by nurse mentors who had learned to deliver it in the pilot study (Ingoldsby et al. 2013) and in a subsequent small-scale implementation effort designed in part to develop a corps of nurse mentors. We hypothesized that sites assigned to the RI would have significantly greater gains in participant retention and completed home visits over time than those assigned to the NFP-as-usual condition and that RI effects would be greater for sites with initially lower rates of retention.

Methods

Participants and Design

As shown in Fig. 1, we recruited 26 sites from two states, focusing on sites that revealed a need and interest in improving their rates of participant retention and completed home visits. They were stratified within states and within high or low retention classifications within those states and then randomized by a study statistician (MM), who had no connection with the sites, to one of three conditions: (1) RI (N = 9), in which nurses were mentored by nurses who had learned the RI during pilot phases of RI development; (2) delayed RI (N = 6), in which nurses were mentored to implement the RI with participants enrolled in the NFP during the third cohort year (described below); or a (3) control condition (N = 11), NFP as usual. We created the DRI condition to cope with a limited number of nurses available to mentor nurses at the beginning of the trial. Volunteer nurse mentors from the RI condition guided DRI nurses to learn the RI with cohort 3. Randomization was conducted at the level of sites rather than individuals to reduce horizontal diffusion of the RI to control participants.

Fig. 1
figure 1

Flow of sites* and participants in cluster randomized controlled trial of retention intervention (RI)*. To reduce complexity, the flowchart does not separate the sample by state. The number of sites in state A is RI 4, DRI 4, control 8. The number of sites in state B is RI 5, DRI 2, control 3

Full size image

The study was conducted with three cohorts of participants enrolled during 12-month periods: The baseline cohort (1) were those NFP program participants (n = 1878) registered between October, 2003 through the end of September, 2004; cohort 2 (n = 1942) were registered between October, 2005 through the end of September 2006; and cohort 3 (n = 2149) were registered between October, 2006 and the end of September, 2007. The baseline cohort preceded cohort 2 by 2 years in order to reduce the likelihood that the baseline cohort in the RI condition would receive the RI at some point during their 2+ years of enrollment in the NFP, as this would attenuate the estimate of change over time for the RI sites. Note that nurses in the DRI condition were not mentored to deliver the RI until they began serving cohort 3. The analysis focused on intervention-control differences in change over time in retention and home visits. Examining changes over time by site increases statistical power and controls for site effects.

One site assigned to the RI condition closed during the second cohort year and did not participate in the mentoring; a second assigned to the RI declined to participate after originally agreeing. Two sites assigned to the DRI chose not to participate, and at a third DRI site, only two of the four nurse home visitors chose to be mentored in the RI. The analysis was conducted as intention to treat.

We assessed retention and completed home visits for the RI group for cohort 3 in addition to cohort 2 in order to determine whether the RI effects were sustained after active mentorship of nurses was completed with cohort 2. Implementation data (described below) were gathered only for cohort 2 participants assigned to the RI condition and for cohort 3 participants assigned to the DRI condition.

Table 1 shows the characteristics of sites in each of the three treatment conditions. As indicated in this table, there were some non-significant differences in organizational auspices by treatment assignment, as might be expected with small numbers of sampling units.

Table 1 Characteristics of sites by condition
Full size table

Table 2 shows the background characteristics of the participants by treatment condition in each of the three cohorts. To be eligible for the NFP program, mothers must be first-time mothers and low-income (usually indicated by Medicaid eligibility). As noted in this table, there were missing data on some participant background characteristics. NFP participants, as indicated in cohort one, were young (M = 19.60 years), had low education (M = 11.23 years), were unmarried (84 %), and the majority were non-Hispanic White (63 %), Black (17 %), Hispanic (16 %), and Native American (9 %). While there were some differences among treatment conditions for some of these background variables, as noted below, controlling for these differences did not change the results.

Table 2 Characteristics of participants by treatment condition and cohort—means and standard deviations (for continuous variables) and percentages (for dichotomous variables)
Full size table

NFP participants consented at enrollment for their data to be used for evaluation and program improvement purposes. Participants were not informed about, or re-consented, for the current trial, which consisted of an innocuous intervention. Nurses assigned to the RI and DRI consented to be mentored in the RI after their sites had been assigned to the intervention conditions, as the RI required additional time to learn, and nurses were asked to gather additional data as part of the intervention. Some nurses consented to participate in focus groups to help us understand aspects of intervention delivery. The study was approved by the Colorado Multiple Institutional Review Board.

Retention Intervention (RI)

The RI has been described earlier (Ingoldsby et al. 2013). It centered on encouraging nurses to have explicit discussions about adjusting the frequency of visits and duration and content of the program to meet the needs of newly enrolled participants. In addition, nurses explained that mothers had the option of switching nurses. Nurses revisited participants’ visit preferences at child age 6 weeks and at 4 months, given that these time periods correspond to periods with relatively higher rates of dropout from the program. To deepen nurses’ skills in client collaboration, addressing clients’ ambivalence about remaining in the program, and adjusting program content to meet clients’ concerns and aspirations, nurses were provided augmented training in motivational interviewing (Miller et al. 2008).

RI sites began mentoring and implementing the intervention with cohort 2 families in October 2005; DRI sites began mentoring and implementation with cohort 3 in October 2006. Nurses at the RI sites participated in 12 1-h structured teleconferences, led by NFP nurse mentors who had learned the intervention in the pilot phase (Ingoldsby et al. 2013) and in a small-scale implementation phase that preceded this trial. Nurses in the DRI were mentored by nurses who learned the RI during the second cohort year. Nurses completed readings, watched MI training videotapes between teleconference calls, and were asked to implement the RI with all newly enrolled participants as mentoring proceeded.

During mentoring sessions, RI and DRI nurses were guided in learning and delivering the RI components, using written RI guidelines, which included scripts, strategies, and case examples. Nurses from each of the sites presented case examples and challenges they experienced in delivering the RI. After each call, teams were sent detailed summaries for their review. At two points during the 12-month intervention period, nurses were provided quantitative reports on their implementation of the RI, on client choices, and on rates of retention (described below).

Retention Intervention Fidelity

Nurses completed an implementation data form during their first year of RI implementation that captured whether the RI was implemented and clients’ choices for visitation schedule. Feedback to sites regarding their implementation of the RI, rates of retention, and completed home visits was shared during the mentoring sessions.

Control Condition

Comparison sites delivered the NFP program as usual.

Measures

Participant Reach

Nurses in the RI and DRI conditions completed an implementation data form for cohort 2 (RI only) and cohort 3 (DRI only) indicating the date of each RI discussion and the components of the intervention that were delivered. Each newly enrolled case in the RI/DRI sites was assigned a value which indicated whether the nurse delivered the intervention with participants during the first intervention year, a variable labeled participant reach.

Participant Choices

After each RI discussion with participants enrolled during the first year of implementation (cohort 2 for the RI group and cohort 3 for DRI), nurses asked participants if they preferred the program to be delivered (a) according to the typical schedule, (b) with reduced visit frequency, (c) with increased frequency, (d) with for a temporary leave from the program, (e) ending participation, or (f) by switching nurses. Responses were recorded on RI implementation data form.

Demographics

Participant age, gestational age, race/ethnicity, marital status, education and employment status, household income, and Medicaid status were assessed at enrollment as part of the standard data gathering for all NFP participants. There are some missing data on participant background characteristics. The exact numbers of cases on which background variables are available for analysis for each cohort and treatment condition are enumerated in Table 2.

Site Level Characteristics

Some of the site characteristics (e.g., length of NFP program operation prior to the “pre-intervention cohort”) were calculated using participant data aggregated at the site level. We used the 2000 US Census to determine the percentage of the population in counties served by the program that lived in urban areas (United States Census Bureau n.d).

Retention and completed visit outcomes were derived from data entered into the NFP web-based information system, into which NFP sites enter program implementation data. There were no missing data on completed home visits and participant retention.

Retention Outcome

Participants were considered to have been retained through the last home visit completed prior to the child’s 27th month of life. Note that this definition differs from that employed in earlier studies of participant retention in the NFP, in which participants’ drop status was derived from nurses’ coding participant drop status, with an original focus on amenable attrition (not caused by participant moves from the catchment area) and augmented by visit completion data, i.e., those without a visit within 180 days were considered to have been dropped, even if nurses did not code the case as a drop (Korfmacher et al. 1999; O’Brien et al. 2012; Ingoldsby et al. 2013). We invoked this change as we were concerned that the original definition was subject to treatment-related measurement bias, i.e., that nurses in the RI and DRI conditions may code reasons for attrition differently after completing mentoring in the intervention than their control-group counterparts. The operational definition employed in the current trial removes this possible source of error and is conservative in that moving from the catchment area is not amenable to nurses’ efforts to engage participants and is ignored in the current definition. Data on completed visits were derived from the NFP web-based information system.

Number-of-Completed-Visits Outcome

After every attempted or completed home visit, nurses filled out an encounter form which included information on whether the visit was completed. From data entered on this form, we calculated the number of home visits completed prior to the child’s 27th month of life (or expected 27th month of life calculated from the mothers’ gestational age at registration for those who stopped receiving visits prior to the child’s birth). The count of home visits was based upon every case registered in the NFP in each of the cohorts. While one might present the completed visit outcome as a percent of expected visits, expressing the outcome as number of completed visits allows more direct comparison between implementation metrics in NFP replication with the original trials of the NFP and with other home visiting programs.

Number-of-Expected-Visits Covariate

Individual participants registered in the NFP at different gestational ages and earlier registration strongly predicts more completed visits. Rather than controlling for gestational age at registration, however, we controlled for expected number of visits given women’s gestational age at registration. Expected visits account for how the standard number of expected visits varies by program phase (4 weekly visits immediately following registration, every other week until delivery, every week for the first 6 weeks following the birth, every other week until the baby is 21 months, and monthly visits until 24 months). The number-of-expected-visits covariate was constructed taking into account fetal losses and maternal and child deaths, which the retention intervention is highly unlikely to affect.

Quantitative Data Analysis

We examined two separate multilevel statistical models that reflected the two intervention conditions (RI and DRI) and the number of years during which the RI and DRI interventions were conducted (one or two). The first model consisted of the following fixed effects: treatment (two levels: RI + DRI vs control), year (two levels: baseline vs first year in which the intervention was conducted, specifically cohort 2 for RI, cohort 3 for DRI, cohort 2 for control), the interaction of treatment and year, plus expected number of visits and state as covariates. The second model was similar to the first but did not include the six sites in the DRI condition and added a three-level intervention year factor (baseline, cohort 2, cohort 3 for both RI and control groups) to examine the enduring impact of the RI.

In the first statistical model, primary analyses contrasted the RI + DRI sites versus control sites, examining treatment differences in the change of retention and completed home visits between the baseline cohort and the cohorts during which the first year of intervention was conducted (cohort 2 for RI and control, and cohort 3 for the DRI). In the second statistical model, we had an additional cohort to test the durability of the RI. We focused the analysis on the contrast of RI-control changes in retention and completed visits from the baseline cohort to cohort 3.

We examined the hypothesis that RI effects would be more pronounced among sites with initially lower levels of retention but dropped the high-low classification factor from the model after finding no significant results or pattern of changes consistent with this hypothesis.

We examined models that included covariates to control for baseline differences in some additional background variables (maternal age, race, Medicaid enrollment, and marital status) but did not include them in the final models because adding the covariates had almost no effect on the estimates and tests and because adding more covariates increased the amount of missing baseline data. The final analyses were conducted as intention to treat, including all NFP participants enrolled in all sites that agreed originally to participate in the trial, irrespective of sites’ and nurses’ actual participation.

The Cox proportional hazard model was used to estimate the hazard of attrition (the reciprocal of retention). A robust sandwich estimate aggregated at the level of sites was used to calculate contrasts of interest (Lin and Wei 1989). The continuous dependent variable (number of home visits completed through the child’s 27th month of life) was analyzed with the mixed effect model with sites as random effects. Findings are reported as adjusted hazard ratios or adjusted (least square) means controlling for the effects of other terms in the model, with standard errors, and two-tailed significance tests.

Qualitative Data Analysis

Given relatively low rates of RI implementation (reach) at some sites described below, focus groups were conducted by one of the investigators (EI) at eight of the NFP sites in the two intervention groups chosen to represent three levels of reach: low (50–64 %; n = 3), medium (65–80 %; n = 2), and high (>80 %; n = 3). A total of 49 participants (41 nurses, 8 supervisors) participated in the focus groups (with a range of three to nine participants at each site). These 60-min sessions were audiotaped for later coding. All nurses and supervisors who participated in the focus groups completed informed consent.

Focus group questions inquired about factors affecting implementation of the intervention and nurses’ motivation and participation in the training (e.g., “Tell us your thoughts about factors or dynamics that might explain why nurses used the intervention at different levels of intensity”) and challenges or barriers that emerged during the intervention (e.g., “what were the challenges for you and your team in spending team professional development time on the intervention?”).

The audiotapes were later reviewed, coded for themes, and categorized using grounded theory (Strauss and Corbin 1990) into the following domains that represented factors affecting implementation: general, nurse, site/team, client, organizational, and intervention-specific factors. Themes/domains were then grouped by site-level implementation rates (low, medium, high) which allowed investigation of patterns of themes and responses across and within implementation groups.

Results

Sample Characteristics

Table 1 shows the site characteristics in each of the three treatment conditions with respect to organizational auspices, duration of program operations, and the degree to which program catchment areas were “urbanized.” While none of the differences among treatment conditions were statistically significant, note that four of the five sites administered through hospitals were in the control group.

Table 2 shows the participant characteristics for intervention and control groups for all cohorts, and the numbers of cases on which information was available for each background variable. Some of the background characteristics differed by treatment condition, but as noted above, inclusion of these baseline covariates did not alter the estimates and tests of intervention effect, so given missing baseline data, they were not included in the final model.

Implementation of the Retention Intervention

Reach

Among all sites assigned to the RI and DRI conditions (irrespective of participation in the trial), nurses reached 63 % of newly enrolled participants in the first year of implementation. As noted above, four intervention sites did not deliver the RI. The mean participant reach among the sites that participated ranged from 53 to 92 % (mean participation = 76 % among those sites that participated); the one site in which two of the four nurses participated in the intervention reached 53 % of the new enrollees.

Participants’ Choices for Program Delivery

Among those reached, 86 % of participants chose the recommended visitation schedule, 10 % chose to decrease the frequency of NFP visits, 2 % chose to increase their visits, and the remainder (2 %) took a temporary leave or ended the program early. One participant requested a change in nurse.

Intervention Impact on Participant Retention and Completed Home Visits

Retention

Figure 2 summarizes the hazard analysis (hazard of dropping from the program) following the first completed home visit to the last completed visit in the RI/DRI and control sites, comparing the baseline cohort with the cohorts enrolled in the sites during the first year of intervention implementation (cohort 2 for RI and cohort 3 for DRI), showing each state separately. Note that state A had substantially higher rates of retention overall (p < 0.001). Averaging across states, control group sites had lower retention in the second cohort compared to the baseline cohort (HR = 1.23, 95 % CI 1.00, 1.50, p = 0.045); there was no difference between the baseline cohort and the cohorts that participated in the RI/DRI sites during the first year of intervention implementation (HR = 1.02, 95 % CI 0.93, 1.13, p = 0.639). The RI/DRI-control difference in change in retention was a statistical trend (p = 0.099).

Fig. 2
figure 2

Hazard analysis of retention in control versus RI + DRI sites during baseline compared to first year of intervention implementation—cohorts 2 (for RI) + 3 (for DRI), shown separately by state

Full size image

The corresponding proportional hazard analysis that focused on the RI-control contrast between the baseline cohort and cohort 3 found a marginally significant decline in retention among control sites between the baseline cohort and cohort 3 (HR = 1.14, 95 % CI 0.1.00, 1.31, p = 0.054) and a non-significant increase in retention in the RI condition (HR = 0.93, 95 % CI 0.82, 1.06, p = 0.277). The RI-control difference in change was significant (p = 0.017).

Completed Home Visits

Figure 3 shows the estimates of completed home visits in the control group and the RI + DRI combined, at baseline and in the first intervention year (cohort 2 for the RI and cohort 3 for the DRI), again for each state separately. In the control group, averaging across states, there was a decline in number of home visits from baseline to cohort 2 (mean = −2.027, SE = 0.942, p = 0.031), and in RI + DRI, there was a non-significant increase in home visits from baseline to the first year of intervention implementation (mean = 0.681, SE = 0.683, p = 0.319), leading to a significant treatment difference in change (mean = 2.708, SE = 1.164, p = 0.020). Again, the overall difference between states was highly significant (p < 0.001).

Fig. 3
figure 3

Mean number of completed home visits for control and RI + DRI sites during baseline versus first year of intervention implementation—cohort 2 (for control and RI) + 3 (for DRI), shown separately by state

Full size image

The corresponding analysis of completed visits that focused on the RI-control contrast between the baseline cohort and cohort 3 (enrolled during the second year of RI implementation) found a marginally significant RI-control difference in change (mean = 2.40, SE = 1.24, p = 0.052).

Association of Reach with Completed Home Visits

Figure 4 shows the relationship between reach and change in completed home visits from baseline to the first cohort in which the RI + DRI was to be delivered. Three of the four sites that had no reach had a loss in completed home visits, but one showed a dramatic gain, the largest of any of the sites in the trial. Three sites with 53–64 % reach exhibited a loss of home visits over time, while five of the eight sites with >64 % reach had gains in completed home visits.

Fig. 4
figure 4

Change in completed home visits among RI and DRI sites between baseline and first intervention year (cohort 2 for RI) and (cohort 3 for DRI) by degree of site reach

Full size image

Qualitative Study of High- and Low-Reach Sites

Nurses in the qualitative study, irrespective of their level of reach, indicated that the intervention was difficult to learn. They reported that the curriculum was too academic and not sufficiently geared to their practice. Medium and high-reach sites (those with >64 % reach) reported that they decided that they would overcome the limitations of the intervention education and mentoring model by working as teams to learn it. These higher reach sites reported that their supervisors were crucial in leading them as a team to learn the RI intervention; their supervisors created a team climate that valued nurses’ lifelong professional development. Those sites with the lowest levels of reach reported that learning the intervention was so challenging, and the time required so significant that they felt overwhelmed. They reported that having to learn the intervention interfered with their time to conduct home visits.

Discussion

Sites assigned to the control condition experienced declines in retention and completed home visits over time, while those that received the intervention did not. This pattern of results is consistent with that observed in the pilot study of this intervention (Ingoldsby et al. 2013) and is likely a reflection of participating sites being in need of help with increasing retention and home visits. The limited increases in retention and completed home visits in the intervention conditions is not surprising, given that 4 of the 15 sites assigned to intervention conditions did not participate, and among those that did, the intervention was delivered to 76 % of the possible participants. The durability of RI impact for cohort 3 found in the RI condition, that is after the RI mentoring ended, suggests that once nurses learn this approach to engaging families, it is carried over to new participants. It is revealing that high-reach sites attained the greatest increases in participant retention and completed home visits. It also is revealing that low-reach sites had almost no changes in completed home visits. Whatever gains in retention and home visits found for the RI were produced by the high-reach sites.

A central goal of the RI was to reinforce with nurses the self-efficacy and strength-based clinical underpinnings of the program, particularly, that NFP participants are the experts in their own lives and that they know the frequency of visits and duration of participation that will work best for them. The results from this trial, as well as the preceding pilot study, align with other studies demonstrating the positive impact of motivational strategies to engage families in mental health treatment (e.g., Grote et al. 2008). They also align with recently published findings from the Memphis trial of the NFP that examined latent class trajectories of home visits completed from pregnancy through child aged 2, in which there was a group of lower risk participants that managed to improve their own health and the health of their children with far fewer visits completed after birth than is included in the standard visitation schedule (Holland et al. 2014). The results of the current trial and the findings on completed home visits in Memphis emphasize that NFP program dosage is best adjusted to families’ needs and abilities to participate.

In some ways, it is remarkable that intervention-control differences emerged given that four of the 15 intervention sites did not participate in the intervention mentoring and given variability in reaching participants among those sites that did participate. While nurses’ overall reach was relatively high (76 %) among sites that were mentored, three intervention sites implemented the intervention with only half of eligible participants. This observation and the results of the nurse focus groups have reminded us how crucial it is to formulate nurses’ professional development education in learning program innovations in ways that are feasible and align with their clinical practice. This, along with nurses’ feedback from the focus groups indicating that the RI was hard to learn, has led us now to introduce the RI as part of core education in the NFP model and to more explicitly align it with their practice.

Given that the three low-reach sites cited lack of teamwork and organizational support for their learning, the NFP National Service Office has taken a more active role in supporting implementing agency administrators’ understanding of the importance of nurses’ professional development as core elements in their education in the NFP model. This lesson has broad implications for the introduction of all new program innovations.

We also note that the number of sites included in this trial is small, a feature of the design that was exacerbated by our need to divide the intervention group into two groups, delaying the intervention for some sites, and our conducting the trial in two separate states with different levels of success in retaining families and completing home visits. The small number of sites increases the likelihood that random assignment will incur some lack of balance in sampling unit characteristics, as we found with the non-significantly larger proportion of control sites being implemented through hospitals. Analyses of implementing agencies’ auspices in the entire set of NFP sites nationally found that hospitals had higher rates of completed home visits than other agency types, so whatever unbalance exists in the current trial is likely to bias against the hypothesis of intervention impact. Future trials employing clusters of sites should ensure larger numbers of sampling units, especially when designs have these additional levels of complexity.

While the pattern of results for the intervention and control conditions was essentially the same for sites in each of the two states, the dramatically different rates of retention and completed home visits by state deserves consideration. All sites in state B were administered through a single state agency, and there was considerable pressure brought to bear on nurses’ completion of specified numbers of visits each week, possibly undermining their motivation to spend time tracking hard-to-reach families and taking time to learn new methods of delivering the program. The intervention’s focus on client-directed visit frequency also may have contributed to administrators’ and nurses’ concerns about nurses’ meeting state-directed productivity requirements. An intervention designed to increase the number of home visits by increasing participants’ explicit control of dosage may appear to be counterintuitive to administrators who are driven by requirements that nurses complete a set number of visits per week. We intend to examine this in greater depth in the future.

Despite the limitations noted above, these findings align with the findings of a preceding pilot study of this intervention (Ingoldsby et al. 2013). And, as noted above, they have led to alterations in core NFP nurse education and program guidelines to support more explicit participant choice in visit scheduling and program content.

This trial, a form or rigorously evaluated quality improvement, was conducted while changes were occurring in retention and completed home visits over time. Nationally, numbers of completed home visits declined over the early years of NFP community replication reaching a low point in 2006 (mean = 23.44, SE = 0.19) but increased to a mean of 24.02 (SE = 0.15) in 2010, which is likely, at least in part, a reflection of the RI intervention findings being communicated to all NFP sites nationally. This diffusion appears to have happened rapidly in state A, which achieved a mean of 31.25 (SE = 0.48) completed home visits by 2010. While some gain was observed among all sites in state B as well, the gain was less dramatic; in 2010, sites in state B completed a mean of 19.99 (SE = 0.45). State A sites were completing visits at about the same rate as those in the original randomized controlled trials (~31), while all sites in the USA were completing about 77 % of those in original trials.

While more is required to bring NFP community replication sites closer to the mean number of completed home visits achieved in the original trials, the current trial and the preceding research on participant retention (O’Brien et al. 2012; Ingoldsby et al. 2013) appears to be helping improve this aspect of NFP implementation in community replication.