Introduction

The epidemiology of mental disorders and their risk factors has been one of the objectives and challenges for modern psychopathology both for adults and children/adolescents. To achieve this purpose, an adequate sampling is necessary to guarantee a high participation and a low probability of drop out to avoid consequent underestimation of the prevalence and determinants for psychopathological problems [1]. However, participation indices are quite variable, and some studies in the literature indicate that they frequently oscillate between 50% and 90% [2]. This has caused the publication of parallel analyses to determine what characteristics are associated with the lack of collaboration, especially in prospective designs with middle and long-term follow-up.

A wide longitudinal epidemiological study in Holland with an adolescent population found that there are no differences in the psychopathological state of participants and non-participants, though it was also made evident that young people with poor academic achievement tend to answer less [3]. In that research, it was observed that when the psychological and functional adjustment of the children is poorer, families tend to accept the collaboration later, and researchers needed to make greater effort to obtain involvement. Other studies have also revealed that the possibility of participation is decreased in lower socioeconomic levels [4, 5], in those with a poor education [6, 7], low salaries [8], or families that require some type of Social-Security benefit [9]. The lack of collaboration is also emphasized for younger individuals [4, 10], males [4, 7, 11], patients with serious behavioral disorders [12, 13], and with the perception of a worse physical and/or psychological health [4].

Furthermore, a number of recent analyses examine the important biasses that are involved in the lack of participation and non-response in epidemiological studies [14, 15], especially in longitudinal designs [16]. Hofler et al. [17] warn that the magnitude of this bias depends on the degree of association between the determinant of the non-participation or drop out and the respective parameter of interest. However, no consensus exists on the techniques that should be employed to determine the degree of similarity between the participants and those who decline to collaborate in the studies and, consequently, there is also nor consensus on the strategies that should be incorporated to correct the bias [13, 18–20]. Though the results found until now are provisional, various studies have shown that there is a direct relationship between the efforts made to recruit the participants and the improvement and precision achieved in the indices that measure the frequency of events and their risk factors [21]. Thus, making telephone calls at different hours of the day and week, sending letters by mail, making personal visits and clarifying the cost and benefit that the collaboration brings can increase the ratio of participation by more than 10% [3]. Offering monetary incentives also increases involvement in the research, as do short interviews (or questionnaires) free of complex words/sentences or compromising questions [8].

Studies referring to the lack of participation and to drop out are still limited (and sometimes contradictory) in the area of infant and juvenile psychopathology. To date, no research has valued these aspects in studies performed in communities with greater vulnerability (for example, poor suburbs). The main objective of this study is to identify social, demographic, and clinical variables related to the collaboration of families in a longitudinal epidemiological study of mental health, conducted in a high-risk population. First, the determinants that predict the decision to participate in the research are assessed. Second, adolescent and preadolescent features, environment and changes that have occurred since the beginning of the study are measured as statistical predictors of drop out.

Method

Participants

The data of this study are part of a longitudinal epidemiological study on mental-health in a high-risk population of children and adolescents [22]. The study was carried out in Badía del Vallès, a small municipality located in the periphery of Barcelona. This municipality is considered to be a typical poor suburb surrounding a big city in Spain.

The study began in 2002. A 3-year follow-up was performed for two cohorts of all children born in 1989 and 1993, respectively. Both cohorts belonged to the Census of the poor suburb in 2001. Each individual was assessed three times (2002, 2003, and 2004), with an interval of one year.

The Census of Badía del Vallès in 2001 listed 147 children born in 1989 (adolescents 13 years old at the beginning of the study) and 121 born in 1993 (preadolescents 9 years old at the beginning). The initial purpose of the research was a prospective psychological assessment of all individuals in this Census (N = 268). To do so, different strategies were used to facilitate the continued collaboration of the families: making telephone calls at different hours of the day and week; visiting the families’ homes when there was no other possibility of contact; publishing an informative article about the study in the neighborhood-association magazine so as to explain in detail the benefits that their collaboration would entail; providing the parents with an oral report showing the results of the psychological assessment as well as indicating the need for professional consultation if indicated; and, an economic remuneration after each assessment.

Measures

A ‘Brief Mental Health Screening Questionnaire’ (BMHSQ) [23] made it possible to obtain basic information about the general state of children in six crucial areas: sadness, complaints from the teachers about inattention, low school performance, problems at home due to disobedience, excessive worries, and problems with friends. In this study, each area has been analyzed in a binary way (0 = adequate behavior; 1 = dysfunctional behavior). Additionally, a total score based on the sum of the six areas was obtained.

The ‘Diagnostic Interview for Children and Adolescents’ (DICA-IV) [24], translated into Spanish (EDNA-IV) [25–27] provided a psychological evaluation based on the DSM-IV criteria [28] for the most frequent diagnoses in juvenile and infantile populations: disruptive behavioral disorders, use-abuse of substances, depressive disorders, anxiety disorders, elimination disorders, eating disorders, and tics disorders. It also made it possible to obtain quantitative indicators such as the number of disorders and symptoms (interiorized, exteriorized, and total). There are three different versions of this interview: for children aged 8–12 years old, adolescent between 13 and 18 years old, and for parents. The variables included in this study were obtained by combining the data of both informants (parents and children) at the level of symptom (a symptom is present if the parent or child reports it). This “disjunctive” (“either or”) strategy supposes that endorsement by any source is sufficient to identify a case as meeting a criterion, and it is the most widely used in clinical practice and epidemiological research [29, 30].

The ‘Global Children’s Assessment Scale’ (CGAS) [31] was used to score the global functional-impairment level produced by the symptoms. This scale provides a score on a range of 1–100 (higher values are indicative of better adjustment). The score is directly selected by the interviewer at the end of the assessment session with the individuals. It is considered that scores lower than 70 are indicators of significant psychological impairment. In this study, we have analyzed the lowest score obtained from parents’ and children reports. The scale has demonstrated adequate reliability and validity [32]: the intraclass correlation coefficients (ICC) valuing test–retest reliability were between 0.44 and 0.65, the ICC valuing between-interviewers reliability were around 0.80 and area under ROC curve valuing the accuracy for discriminating between healthy and psychiatric children was higher than 0.80.

The ‘Child Behavior Checklist’ (CBCL) [33] was used as a dimensional measure of the general state of children from the perception of their parents. CBCL contains 113 items designed to assess 6–18 years olds’ feelings and behaviors. Parents answer 0 (not true), 1 (sometimes true), or 2 (always or almost always true). In this study we analyzed scores for three scales: externalizing problems, internalizing problems, and total score.

The ‘Schedule for Risk Factors’ (SRF) [34] is a wide semi-structured interview that provides information about factors pertaining to the child, the context and the family that could have an impact on the actual psychological status of children and adolescents. Parents’ and children’s versions have been analyzed, selecting the highest risk level. In this study, we have selected variables included in the next sections: socio-demographical features, children’s medical history, school and other activities, family history of psychopathology, life events, rearing style, and use of services. Moreover, concrete data in sociodemographical section relating to parent’s studies level and profession were combined according to Hollingshead index [35] for obtaining the Socio-Economic-Status of families (SES).

Procedure

The staff of the primary Mental-Health Service of Badía del Vallès participated in the recruitment of families. Initial contact with the families was made, as much as possible, by telephone. The characteristics of their collaboration were explained as were the benefits were they to accept. When telephonic contact was not possible, a social worker personally visited the families’ houses. Parents who rejected participation were required to answer the BMHSQ (generally over the telephone), family’s socio-demographic data and the reason for declining the collaboration.

Informed written consent was obtained from parents and oral consent from children in those families that accepted participation. In the assessment sessions, the diagnostic and psychological information was obtained from the children themselves and their parents. The interviews were conducted separately and simultaneously with both informants by expert interviewers. All previously described measurements were recorded in each assessment.

Some days after each assessment session, and whenever it was possible, families were informed about the presence or absence of psychological problems in the child, and about the possible need of receiving professional help.

Families that participated in any assessment but finally decided to drop out were required to answer the BMHSQ, generally via telephone.

Statistical analysis

Statistical analysis was carried out with SPSS 13.0. First, socio-demographic and functional variables registered at the beginning of the study for the set of children in the 2001 Census were related with the type of participation (complete, partial, or none). Multinomial regression models were used (FSTEP test method). This methodology is similar to the classical binary logistic regression, but is more general since the dependent variable is not restricted to two categories. Its parameters are interpreted in terms of odds ratios (OR). The demographic and functional features were included as predictors; the type of collaboration was included as response.

Next, children and their families’ clinical characteristic, measured in the first and last assessments, were related to the drop-out rate. Cox’s regression models were used for the analysis of the initial data and logistic regression models were used for the final information. In the Cox’s models, the response was defined as time of participation (in years) for each case (that is, the time until they drop out or end the research, ‘survival time’). In logistic models, the criterion was the decision to maintain collaboration or to drop out. The independent variables for both types of models were the global indicators of psychopathology and functioning measured with the EDNA, SRF, CBC, and CGAS. The information from the first assessment was analyzed to estimate the predictive capacity of the children’s initial state. To assess the prognostic value of the state previous to dropping out, the indicators analyzed were those registered in the last assessment for families that left; for families collaborating over the 3 years, the indicators analyzed were those features obtained in the second assessment. Since the decision to drop out is related to children’s age, the cohort was included as a control (adjustment) variable.

Next, the similarity of the distributions of the global functioning variables registered through the BMHSQ in the last assessment session was examined by comparing the families that dropped out to those that did not. The objective of this analysis was to verify whether the general state of the children at the specific time of the assessment was a statistical indicator of the immediate decision to leave or to maintain the collaboration. Binary logistic regression models were used for comparing the state of the children who dropped out (in any one of the follow-ups) and those who did not. The cohort was again considered to be an adjustment variable.

Finally, in order to discover other determinants of non-collaboration, proof was sought for whether the changes registered in the BMHSQ between the first and last assessment were additionally indicative of dropping out. The groups selected for this analysis were also families with partial and complete participation. Measurements analyzed correspond to the last assessment session available. Logistic models were used and defined the cohort as a control variable.

Results

Participation and dropouts

Figure 1 presents the recruitment and participation plan of the study. In the 1993 cohort (preadolescents), 65 children (53.7%) participated in the entire study, 7 (5.8%) collaborated partially, and 49 (40.5%) refused to collaborate. In the 1989 cohort (adolescents), 61 youths (41.5%) attended the three assessments, 18 young people (12.2%) abandoned before ending the study and 68 (46.3%) did not wish to participate. These data show no statistical association between the type of participation and the cohorts’ age (P = 0.061). Nevertheless, there were significant differences between both age groups and the reason given for non-participation (P = 0.036). In families with preadolescents, the most usual cause adduced was lack of interest (50%), followed by the conviction that their children were healthy (32.1%). An important number of parents of adolescents also alleged lack of interest in the research (40%), but other significant proportion alluded to lack of time (22.5%) or other motives for non-collaboration (25%) such as current attendance of mental-health services.

Fig. 1
figure 1

Process of participation and drop out

Full size image

Features related with the families’ participation decision

Columns 1–5 in Table 1 report the description in each group of the main socio-demographic and clinical features at the beginning of the study. The remaining columns include the results of multinomial regressions that value the association of these characteristics with the type of participation. These results indicate that non-participation probability increases in families with low socioeconomic levels, belonging to minority cultures, requiring the help of social services and with children with poor academic performance. This probability is also higher if the father is unemployed or requires Social-Security benefit. The comparison between dropouts and non-participants at the beginning of the study indicates that it is also more probable for families to initially reject collaboration if the mother is unemployed or requires social assistance; in contrast, families decline an initial commitment when it is the father who is out of work.

Table 1 Socio-demographic and clinical features related with type of participation
Full size table

Predictors of dropouts

In the preadolescent cohort, the risk of dropping out for families that accepted participation at the beginning was 9.7% (95% CI: 4–19) and 22.8% (95% CI: 14.1–33.6) in the adolescent cohort. These data indicate a strong relation between the age of children and the probability of dropping out (P = 0.031; RR = 2.3, 95% CI: 1.04–5.3).

Columns 1–4 in Table 2 report the results of Cox’s regressions for the association between the first diagnostic assessment and the rate of drop out. These data show that predictors of increased drop-out rates are children’s use of mental-health services and need for learning-support sessions. In contrast, families with children who watch more TV present a lower drop-out rate.

Table 2 Predictive value of the clinical features measured in the first and last diagnostic assessments on the probability of drop out
Full size table

Columns 5–8 in Table 2 report the results of logistic regressions for the association between the reports obtained from families in the last assessment and the decision of not participation in the following year. These data indicate that more psychopathology in the last diagnostic assessment, being physically unhealthy and a higher number of life-events are predictors of dropping out. A lengthy interview without return of reports to the families is also associated with higher odds of leaving the study.

A statistical association between measurements of general functioning obtained in the last assessment session (via the BMHSQ) and the probability of giving up has not been found, nor was one found between the changes registered in these variables and the ceasing of collaboration.

Discussion

The participation ratios of 53.7% and 59.5% for both cohorts in this study are quite good, and similar to those obtained in other epidemiological studies with infantile and adult cohorts [2]. It must be emphasized that we have being working at a general (or whole) population level, that is, with all children in the municipal census of children born in 1989 and 1993. These participation proportions are even more valuable if we consider the socio-cultural features of sample, since other studies in the literature showed that the possibility of participation decreased for lower socioeconomic levels [5] and poor education cohorts [6]. On the other hand, the drop-out proportion was relatively small (5.8% of preadolescents and 12.2% of adolescents). In this regard, the strategies and efforts used to obtain the families’ involvement can be considered effective, particularly if we considerer that it is particularly difficult to attain adherence to the study by adolescents from whole populations.

The cohort’s age is not related with participation type. This result differs from other studies that suggest more probability of involvement at the beginning of the studies when individuals are older [4, 10]. This discrepancy could be explained by the under-powering of the statistical procedures in this study due to the small sample. Other reason for this discrepancy could be that the age range in this research is lower. In fact, independent analyses of the initial decision of participation ‘versus’ non-participation (without considering whether collaboration was total or partial) reveals that in the 9-year-old cohort 40.5% of the families refused to be involved in the study, as compared to 46.3% in the 13-year-old cohort. This difference may have increased in the case of cohorts having a greater age range. On the other hand, one of the main arguments for refusing to collaborate at the beginning of the study was the lack of interest, which is coincident with the results obtained by Kotaniemi et al. [36], and indicates that it is necessary to design new strategies in order to improve the motivation of families towards this type of research projects. It is important to provide adequate information regarding the importance of studies for the entire community, and the specific benefits that their collaboration will imply.

Regarding the socio-demographic features related with the lack of collaboration, results are in concordance with the literature: the most unfavorable family situations, such as low socioeconomic status, minority cultures, unemployment or the need of Social-Security benefits are predictors of non-participation [4, 5, 9].

Children’s functional performance (measured with the BMHSQ) was not a statistical predictor of participation, except for failures at school (the same was observed in Winter et al. 2005). This is a remarkable result, since it could be hypothesized ‘a priori’ that collaboration would be facilitated when parents perceive any type of impairment in their children. However, the result can be interpreted differently: either these families do not consider that their involvement in the research is useful to them (this could also justify the high proportion of cases with disinterest) or else they are involved in another clinical assessment at the time. This latter situation seems very probable, since 16.2% of families declined to participate citing different reasons for their lack of interest: lack of time or the perception that the child is healthy. This ratio is similar to the proportion of people who attend mental services found in the first assessment between participants (21% in the preadolescent cohort and 12% in the adolescent cohort).

The child’s age was found to be an indicator of drop out with the risk of ceasing collaboration 2.3 times higher in the adolescent cohort (in comparison with preadolescents). One of the explanations for this fact could be that as adolescents are older, their opinions are more important in the final decision made by the families. The reinforcements used in this study probably had a greater influence on the parents and smaller children, but not on the adolescents. In this regard, it is indispensable both to anticipate and to address efforts aimed at all subject collaboration and commitment, especially when the individuals are adolescent.

Consultation with mental-health services is a predictor of dropping out. An explanation could be that people who decide to request professional help consider that participation in these studies is not necessary or it is a mere duplication of efforts. On the other hand, it was observed that individuals with more psychopathology, bad physical health or more life-events in the last assessment have a higher probability of leaving the study. The literature points to people who attend consultation having a higher number of problems. The relationship would therefore be the following: people with a greater number of difficulties seek professional help and cease collaboration. This datum is very relevant in the area of epidemiology since there is a strong association between the use of services and the state of health (which is, in fact, the objective of study in this research). The consequence on published results would be an underestimation of the presence of psychological disorders.

Finally, certain characteristics of the assessment process have an influence on the decision of continuing or ceasing the collaboration. Very long interviews and the absence of the corresponding return of reports to the families increase the possibility of drop out. This could be due to different reasons. First, the structure of the measurements used in this study (very common in psychopathology) causes the assessment sessions to be slower (and, consequently, longer) in individuals with more problems, since more time is required to explain all the symptoms. Thus, a relationship with a mediator variable is produced: the presence of more pathology increases the duration of the interview and consequently influences the final decision to leave. Second, not returning reports implies that participants’ consider the benefits of their collaboration to be reduced, and therefore, their motivation also decreases.

Limitations

The proportion of subjects of the whole population that declined to participate at the beginning was 43.7%. Once enrolled in the study the rate of drop-outs was very low. The low rate of drop-out hindered the evaluation of the characteristics of those families ceasing their collaboration. Furthermore it prevented the application of more complex and global statistical models for obtaining a person-profile of people tending to drop out of these types of research projects (for example, a cluster analysis). On the other hand, because the small number of dropouts, the absence of relationships with the predictive features included in the study must not be strictly interpreted as real lack of association. However, it should be argued that the observed disadvantage (low rate of drop outs) is a notable advantage for the initial project since it contributes to incrementing the validity of the epidemiological results.

Conclusion

This project has practical implications for reducing the non-participation and drop out of the epidemiological studies into mental health in children and adolescents. The results indicate that the variables increasing lack of collaboration from families with high psychosocial risk are attributable to: (1) social disadvantages (low SES, minority cultures, unemployment, etc.), (2) the health of the child (more physical and psychological problems and the use of mental-health services), and (3) the characteristics of the assessment (lengthy interviews and the absence of final reports for families). Therefore, before beginning a study, special attention should be paid to the effects of these variables. There are three groups that require particular consideration so as to obtain and to maintain their participation in longitudinal studies related to the mental health of children and young people: (1) adolescents, (2) families with lower socioeconomic levels, more economic problems, or pertaining to minority groups, and (3) those with more psychopathologies. Some useful strategies could be: to use peers to involve adolescents and to obtain their participation (for example, colleagues or friends who have previously collaborated), to attend associations or social services that could easily reach people with economic problems and present the study to them in a more direct way. In effect, such measure would recruit children’s psychologists/therapists in the motivation of the families to participate in the assessments (as long as this is necessary). Likewise, it is important to give adequate information about the duration of the assessment and to insist on the possibility that certain interviews might need to be longer, depending on the characteristics of each family. In the end, making use of this information will facilitate the inclusion of the correspondent adjustments during the planning of future research, will reduce the non-response bias and will provide more reliable and precise epidemiological estimations.

Still, lower socio-economic families might require the design of shorter instruments. The field of assessment in epidemiology has made attempts in this direction in relation with screening and it is recognized that such instruments must be short and simple to reduce subject burden and to maximize further participation in later stages. The Short Mood and Feelings Questionnaire [37] or the Strengths and Difficulties Questionnaire [38] are examples. Further efforts must be done for designing epidemiological assessment instruments appropriate for the populations with specific needs.