Introduction

Preterm births account for about 12% of all births in the United States [1]. Infants born preterm have increased risk for chronic health problems, developmental delays, and lagging somatic growth [2,3,4]. Noting this increased risk for poor outcomes, over a decade ago the National Institute of Child Health and Human Development Neonatal Research Network created guidelines for high-risk infant follow-up. Those guidelines recommended that all infants born ≤28 weeks gestational age or ≤1000 grams have periodic, comprehensive health and developmental assessments, while acknowledging that other infants not meeting those criteria might also benefit from follow-up on the basis of perinatal morbidities and therapeutic interventions received [5]. Many neonatology departments with accredited fellowship training programs provide some degree of medical and developmental follow-up for high-risk preterm infants after hospital discharge [6]. High-risk infants also receive follow-up services from community-based primary care pediatricians, subspecialty medical clinics, and early intervention programs, among others. There is, therefore, marked heterogeneity in the patient populations, content, and timing of clinical follow-up, as well as the resources available for follow-up services.

Optimizing referral criteria is crucial for matching services to patient need. Commonly, referral for high-risk follow-up is based on gestational age alone [6]. This practice is limited because although a higher risk for long-term morbidity is associated with lower gestational age at birth, there is wide variability in outcomes among infants born at any given gestational age [2]. This variability depends on individual infant, family, and medical center characteristics [7, 8]. Relying solely on gestational age to identify risk might lead to a mismatch between follow-up service need and provision and could result in a misallocation of resources. Improved referral criteria have the potential to reduce unnecessary expenditure of clinician and parent time and effort.

Although many models of high-risk follow-up are currently in use, most are clinic-based and rely on physical examination by a clinician as well as formal developmental testing by a psychologist, physiotherapist, and/or developmental specialist. While comprehensive, these models are both time-consuming to providers [6] and potentially burdensome to families. Because of this intensity, scheduling difficulties and long intervals between visits may be encountered, limiting the flexibility to respond quickly to infant and family needs.

Better identification of the infants requiring high-risk follow-up care might take advantage of newer approaches to the data collection such as email and text messaging. These technologies have been implemented for monitoring symptoms in areas as varied as pediatric palliative care [9], allergy [10], and attention deficit disorder [11]. An electronic developmental screening tool has been validated for use on mobile devices and shown to be acceptable to community health workers in resource-poor settings [12]. Though not widely-used in clinical practice among high-risk infants and their families, a recent survey of parents of preterm infants revealed high rates of internet connectivity and a willingness to complete developmental questionnaires online [13].

An important gap in the current approach to follow-up for preterm infants is how to identify those most likely to benefit from participation in specialized clinic-based follow-up programs. We therefore propose a novel approach to risk-assessment and referral for clinic-based high-risk follow-up using electronic data collection tools. This study aimed to evaluate the use of the data from emailed parent questionnaires to identify the medical and developmental needs of very preterm infants after neonatal intensive care unit (NICU) discharge. Our objectives were 1) to assess the feasibility and acceptability of emailing parent questionnaires after NICU discharge to assess the health and development of very preterm infants and 2) to examine the extent to which the data from post-discharge parent questionnaires could help identify preterm infants who were most likely to benefit from specialized neonatal follow-up care.

Methods

Setting and participants

Study participants were recruited from the Beth Israel Deaconess Medical Center (BIDMC) NICU. All infants <32 weeks’ gestation who survived to discharge or transfer were eligible. Infants whose parents could not complete questionnaires in English were excluded. Parents of eligible infants were approached about the study once the infant was no longer considered critically ill. After providing informed consent, parents provided detailed contact information to the study team, including email addresses.

Participants were evaluated at 6 months corrected age (CA) in the Boston Children’s Hospital Infant Follow-Up Program. The program is a tertiary children’s hospital-based multidisciplinary clinic providing medical and developmental follow-up for infants discharged from 3 Boston NICU’s—including BIDMC—and community hospital-based special care nurseries. Referral criteria include gestational age <32 weeks, birth weight <1500 g, or clinician preference based on specific diagnoses or illness severity. Routine assessments include a medical evaluation, developmental testing, and a psychosocial evaluation. On the basis of these assessments, referrals are made for subspecialty care (e.g., pulmonary, neurology, orthopedics), Early Intervention, and other community programs. To ensure that all study participants received these evaluations without burdening families with co-payments, the cost was covered by the research study, rather than billed to insurance. Participating families received compensation in the form of gift cards and parking and meal vouchers on the days of clinic visits.

The study procedures were approved by the Institutional Review Boards of BIDMC and Boston Children’s Hospital.

Clinical data

Clinical information was collected from BIDMC NICU Baby Log, an electronic database maintained for all NICU patients. Baby Log contains over 100 variables drawn from the electronic medical record and birth. Specific clinical variables obtained from Baby Log were as follows: birth weight, gestational age, NICU morbidities (infection, gastrointestinal perforation, oxygen therapy at 36 weeks, intraventricular hemorrhage, discharge weight <10th percentile). Illness severity within the first 24 h of life was measured using the Score for Neonatal Acute Physiology (SNAP) [14]. Discharge diagnoses were abstracted by our research team from dictated NICU discharge summaries.

Parent questionnaires

At 44 weeks postmenstrual age (PMA), parents were emailed a questionnaire comprising questions related to infant health, development and family psychosocial well-being. We assessed the quality of preparation for transition from NICU to home using relevant items from the Care Transitions Measure (CTM) [15]. We adapted a feeding difficulties scale developed by Demauro [16]. Scores from the 6 individual domains were combined to create a composite score, with higher values indicating greater feeding difficulty. Questions from the National Survey of Children with Special Health Care Needs were used to determine the presence of special health care needs, defined as use of prescription medication, medical subspecialty care, special therapies, or medical equipment [17]. We used the Edinburgh Postnatal Depression Scale (EPDS) [18] to assess maternal depressive symptoms, with higher scores, indicating more symptoms, and a score >10 indicating risk for clinical depression [18]. Each of these domains was selected based on relevance to at least one component of our primary outcome [19,20,21,22,23]. In this questionnaire, we also ascertained markers of family socioeconomic status, including maternal educational attainment and annual household income.

At 6 months CA (~2 weeks prior to the Infant Follow-Up Program appointment) parents were administered the feeding questionnaire, the special health care needs assessment questions, and the EPDS. Three additional measures were included at this time. The Motor and Social Development Scale (MSD) measures the motor, social, and cognitive development of young children from birth through 3 years (mean 100, standard deviation 15) [24]. The Baby Pediatric Symptom Checklist (BPSC) is a measure of irritability, inflexibility, and difficulty with routines for infants from birth to 18 months of life. Each domain has a score range of 0–8 with a median of 1 for infants at 6 months in a normative population of infants [25]. The Infant and Toddler Quality of Life (ITQOL) is an assessment of physical, mental, and social well-being for children between 2 months and 5 years of age with four subscales and a composite score (mean 100, standard deviation 15) [26].

Just after the clinic visit, we administered a third parent questionnaire via email. This questionnaire included items about the acceptability, perceived value, and perceived burden of attending in-person follow-up visits.

Parent questionnaires were administered electronically by email using REDCap [27], a secure, web-based application designed to support data capture for research studies. Feasibility of electronic questionnaire administration was evaluated by the ability of the study to team to send links to the questionnaires by email and receive parent responses in REDCap. We inferred acceptability about emailed questionnaires from rates of enrollment and survey response.

In-person evaluation

During the clinical visit, a neonatologist or neonatology fellow performed a medical evaluation that included an interval history, developmental and feeding history, and physical exam. Anthropometric measures included weight (kilograms) on an electronic scale, length (centimeters) on a length board, and head circumference (centimeters) with a standard clinical tape measure. Percentiles were determined for corrected age using the World Health Organization growth standards [28]. A pediatric psychologist and physical therapist administered the cognitive and motor subscales of the Bayley Scales of Infant Development-III (mean 100, standard deviation 15) [29]. Scale scores for the infant’s age corrected for age at gestation were calculated from raw scores. Each family also had a social work assessment at each clinic visit. The clinical team discussed each patient collaboratively to determine the need for new referrals, which were recorded in real-time by the research assistant.

Analyses

We first described the sample by simple means (standard deviations), medians (interquartile range) and proportions. In defining the outcome variables, we conceptualized preterm infants who would most benefit from specialized follow-up care as those with the following, assessed in-person at 6 months: (1) Bayley-III cognitive or motor subscales score <85 (representing 1 standard deviation below the mean representing moderate-severe neurodevelopmental delay [30]), (2) weight, length, or head circumference <10th percentile, and (3) new referral at the assessment visit for medical, developmental, or psychosocial services. We then tested bivariate associations of variables derived from questionnaires with each of the binary outcomes (Bayley scores <85, weight, length, or head circumference <10th percentile, or new referral). For these analyses, we used chi-square and Fisher’s exact tests to compare proportions and Wilcoxon–Mann–Whitney tests to compare medians across outcomes. We considered p < 0.05 statistically significant.

Results

Sample characteristics

Infant characteristics are presented in Table 1. The mean birth weight was 1213 g (SD, 402) and median gestational age was 29.0 weeks (IQR 28.0, 31.0). A majority of infants were male (65%) and half were white (52%). Most mothers had earned a college diploma (80%) and more than half had household income greater than $80,000 per year. Those with missing outcomes data did not differ on baseline characteristics from those without missing data except for shorter NICU stay (46 vs. 70 days) and fewer mothers with greater than college education and (65 vs. 76%) and fewer families with annual income greater than $80,000 (36 vs. 62%).

Table 1 Infant and family characteristics
Full size table

The most common diagnosis at NICU discharge was oxygen therapy at 36 weeks’ postmenstrual age 17 (35.4%). Twenty-six (54.2% of infants had at least one NICU complication, with the median of 1 complication (IQR 0.0–1.0). The median length of hospital stay was 67.5 (IQR 46.0–97.5) days.

Feasibility and acceptability

There were 62 infants meeting gestational age criteria identified during the study period (Fig. 1). Of those, 51 (82%) were enrolled in the study. Forty eight families (94%) returned the 44 week PMA survey with 29 required no reminder. Three families did not respond at all and were lost to follow-up at this stage.

Fig. 1
figure 1

Participant recruitment and retention

Full size image

Forty-six families (90%) returned the 6-month CA survey with again 29 without any reminder and the remainder with up to three. An additional two families did not respond at this time point yielding a total of 5 lost to follow-up

Thirty-six (70%) attended the infant follow-up visit at 6 months CA. Six families opted out of the clinical follow-up program, 5 due to concerns such as the time commitment, distance, and missing work. Five families either did not show for their appointments and/or could not be rescheduled. One family was not able to be reached to schedule their follow-up appointment

Due to the possibility that our questionnaire could uncover clinically significant maternal depression, we scored the Edinburgh Postpartum Depression Scale within one business day of its submission. Any woman with a score >10 was called by a clinician (JSL or MBB) to assess symptom severity, risk to mother and infant, offer help, and make follow-up services recommendations. At each assessment point, about 20% of the women had a depression score >10. All were aware of their depression and/or anxiety symptoms and were either engaged treatment or in the process of seeking treatment. There were no safety concerns for any of the mothers or infants noted at either time point.

Questionnaire findings

Responses from parent questionnaires at 44 weeks PMA and 6 months are presented in Table 2. At 44 weeks, the median transition to home score was 92.6 (IQR 81.5, 100), indicating a high level of preparation for transition from NICU to home. Seven mothers (14.6%) had EPDS scores above the clinical threshold of 10. At 6 months, eight mothers (17.4%) had EPDS scores above 10. The median score for overall infant health at 6 months on the ITQOL was 100 (IQR 85.0, 100).

Table 2 Parent-reported measures at 44 weeks postmenstrual age and 6 months corrected age
Full size table

Six month CA follow-up outcomes

At 6 months CA, average scores on the Bayley Scales of Infant Development were 96.2 (SD 10.7) on the cognitive subscale and 93.9 (SD 17.1) on the motor subscale. Eight (22.0%) infants had either cognitive or motor Bayley scores <85 and 15 (41.7%) had at least one size parameter <10th percentile (Table 3). Twenty-four infants (66.7%) received a new referral for medical, psychosocial, or developmental services at the 6-month follow-up visit. The most common new referral was for developmental support services (24, 66.7%).

Table 3 Six month clinic outcomes
Full size table

Longitudinal associations of 44 week PMA parent-reported measures and infant outcomes at 6 months CA

Associations of each 6-month CA outcome, and the composite outcome, with 44-week PMA parent measures are presented in Tables 4a, 4b. Lower transition scores were associated with Bayley score <85 (81.5 vs. 96.3, p = <0.01) and weight <10th percentile (85.2 vs. 96.3, p = 0.05). Higher feeding difficulties scores were associated with weight <10th percentile (1.73 vs. 1.42, p = 0.04). Having a special health care need was associated with new referral at 6 months CA (83 vs. 42%, p = 0.02). None of the parent-reported measures was statistically associated with the composite outcome.

Table 4a Longitudinal associations of parent-reported measures at 44 weeks PMA with infant risk markers at 6 months corrected age (n = 36)
Full size table
Table 4b Cross-sectional associations of parent-reported measures with infant risk markers at 6 months corrected age (n = 36 infants)
Full size table

Cross-sectional associations of parent-reported measures and infant outcomes at 6 months

Associations of 6-month CA outcomes with parent measures at 6 months are presented in Tables 4b. Higher feeding problem scores at 6 months were associated with Bayley score <85 (1.67 vs. 1.27, p = 0.01) and receiving new service referrals (1.45 vs. 1.25, p = 0.05). Maternal depression symptoms were associated with the composite measure of any adverse 6-month outcome (3.0 vs. 1.0, p = 0.04). Poor overall infant health-related quality of life was associated with Bayley score <85 (85 vs. 100, p = 0.02), new referral (85 vs. 100, p = 0.01), and any adverse outcome (85 vs. 100, p = 0.03). Infant Motor and Social Development scale scores and Baby Pediatric Symptom Checklist scores were not significantly associated with any of the 6-month CA outcomes.

Post-visit parent survey

Responses to the post-visit questionnaire are presented in Table 5. The median duration for an in-clinic follow-up appointment was 4.9 (3.0–8.5) h. Most (53%) respondents reported having to take time off from work to attend the visit. All respondents drove to the clinic site and a plurality (37%) traveled over 11 miles to attend the visit. Five families (17%) reported having to arrange child care for other children in order to attend the appointment.

Table 5 Parent-reported experiences of in-person follow-up (n = 30 families)
Full size table

Discussion

We present the first study to our knowledge in which the data about preterm infant health and development was captured using serial post-discharge parent questionnaires administered by email. Our study is an important first step towards demonstrating the value of incorporating parent-reported measures alongside more traditional clinic-based follow-up for high-risk preterm infants. In this study, based on detailed information about parent participation and staff effort, we established the feasibility of this approach in our setting. Additionally, we found relationships of several parent-reported measures with in-person outcomes, demonstrating the promise of parent-reported measures as valuable indicators of infant status after NICU discharge. We also report evidence of substantial parental burden associated with attending a comprehensive in-person follow-up visit. Taken together, these results suggest the utility of the electronic data capture of parent-reported measures for new or existing follow-up programs.

From our high response rates, we infer that electronic administration of post-discharge questionnaires was highly acceptable to families. The majority of families initially approached about the study enrolled, indicating a general openness to staying connected with the care team during the transition home and acceptance of communicating by email. This impression was further reinforced by 90% of parents returning both surveys. Notably, 9 of the 15 families that did not return for in-person follow-up completed questionnaires at both time points. We interpret this finding as indicating the relative ease of electronic communication compared to in-person clinic visits. Using emailed questionnaires may serve as an effective alternative for families unable or unwilling to participate in traditional follow-up programs due to constraints on time or transportation.

Our data regarding staff effort for ensuring questionnaire completion and the safety of those with heightened risk for maternal depression speak to the resources required to implement and maintain a follow-up program that incorporates electronic parent questionnaires. A majority of respondents completed the emailed questionnaires within 1 week of receipt, but some families required multiple email or telephone reminders. The safety plan for parents whose EPDS responses raised clinical concerns about depression hinged on timely contact with the family to ensure physical safety, requiring additional time and effort from the study team. Overall, our results suggest that while there are efficiencies associated with administering parent questionnaires via electronic data capture, there nonetheless remains a considerable degree of staff effort to ensure completion, and to provide timely clinical follow-up for sensitive issues, that must be accounted for.

Pediatric patients and their families have used web-based technologies to access health information [31] and manage chronic conditions [32]. Health services researchers have taken advantage of evolving technologies and expanding electronic connectivity to recruit for and conduct studies on medication adherence, symptom trajectories, and health behaviors [33]. To our knowledge, the use of electronically administered survey instruments for ascertaining medical and developmental risk after NICU discharge has not been attempted previously. Our novel approach to risk-assessment holds promise for further applications in clinical follow-up.

Current practice is often to refer infants for follow-up based solely on birth weight or gestational age [34], despite knowing that many other factors contribute to poor health and developmental outcomes. Our analyses revealed several significant parent-reported indicators of adverse 6-month outcomes in addition to birth weight and gestational age, findings consistent with prior literature [2, 35]. Some of the risk indicators included infant health characteristics extracted from the medical record, such as type and number of morbidities at discharge. Yet several others, like maternal depressive symptoms, infant feeding behaviors, and poor infant health-related quality of life, came to light from parent questionnaires administered after hospital discharge.

A strength of our study is the very high response rate to our parent questionnaires, even among families who did not return for in-person follow-up visits. Another is the use of validated measures of parent-reported infant behavior, health, and development. Despite these strengths, our study has several limitations. There was a high degree of homogeneity among families enrolled in the study, particularly with respect to socioeconomic status and preparation for transition home. The survey instruments were also only available in English, narrowing the diversity of families we were able to include in the study. The low sample variability in these characteristics limited our ability to detect meaningful differences by socioeconomic factors, if present, and also limits the generalizability of our findings. As logistical and financial barriers to attending clinic-based follow-up are likely to be more frequent in populations of low socioeconomic status, and the impact of these barriers on engagement with follow-up care is minimized with our survey approach, further study is needed in diverse settings. While we were able to detect several important associations between parent responses and infant outcomes, we did not have statistical power to develop and test predictive models, including establishing the test characteristics (sensitivity, specificity, positive and negative predictive value) of questionnaire variables individually or in combination.

Our success with parent responses to emailed questionnaires did not carry over to in-person visits, despite extensive staff effort in contacting families, and the offer of financial compensation for attending the in-person follow-up visit. This finding likely speaks to logistical barriers to in-person visits to the medical center, requiring expenditures of time and money for travel and hours taken from work, factors that provided motivation for our pilot study and were reinforced by responses to our post-visit survey.

Medical and developmental follow-up for infants after NICU discharge serves to monitor those at highest risk for poor outcomes and provide interventions to optimize growth and development. The current practice of referring infants based on gestational age ignores the broad variability in outcomes among preterm infants. There are also notable logistical and financial barriers to providing in-person follow-up services to the growing population of NICU survivors. Emailed data collection may prove a feasible alternative or complementary modality for reaching families who otherwise might be lost to follow-up. Additional information on infant health, behavior, and the home environment obtained from electronic parent questionnaires may further augment our ability to identify those infants most likely to benefit from clinic-based follow-up, determine the timing and frequency of visits, and monitor response to medical therapies and developmental interventions in real time. A larger intervention study is required to test the efficacy of such an approach to high-risk infant follow-up.