Adolescence is characterized by several developmental changes including changes in the social domain. One of the most notable social changes is the increased importance of peer relationships. For example, adolescents spend more time with their peers and peers become an important source of social support (Furman and Buhrmester, 1992). However, at the same time, peer relationships become more challenging during adolescence. For example, adolescents develop higher expectations regarding peer relationships (Heinrich and Gullone, 2006) and become more sensitive to their peers’ expectations and opinions (Brown and Larson, 2009). Therefore, not surprisingly, adolescents are particularly vulnerable to developing feelings of loneliness, social anxiety, and depressive symptoms (Alfano and Beidel, 2011; Costello et al. 2003; Qualter et al. 2015) as these are all internalizing problems with strong interpersonal components.

Internalizing problems comprise symptoms such as worry and sadness that are experienced within the individual. The primary internalizing problems are depression, anxiety, social withdrawal, and somatic or physical problems without known medical cause (Gueldner and Merrell 2011). In addition, loneliness is conceptualized as an emotional problem (Matthews et al. 2016) and is, therefore, also commonly categorized as an internalizing problem (Blossom and Apsche 2013). Loneliness is defined as the negative feeling that people experience when they perceive their social relationships as unsatisfying, either quantitatively or qualitatively (Perlman and Peplau 1981). In other words, when people experience that the quality of their social relationships is worse than they desire or when the number of relationships they have does not meet their expectations, feelings of loneliness are likely to emerge. Social anxiety is characterized by a prominent fear of one or more social situations in which the person is exposed to possible scrutiny by others. Individuals with social anxiety worry that they will behave in a particular way or show anxiety symptoms (e.g., blushing or a trembling voice) that will elicit negative evaluation by others. (American Psychiatric Association, 2013). Depressive symptoms in youth, in turn, include the two core symptoms of depressed mood and loss of interest and pleasure in activities, apart from symptoms such as irritability and problems with physical functioning (e.g., sleep difficulties, fatigue, and changes in appetite) (American Psychiatric Association, 2013).

Literature regarding the association among loneliness, social anxiety, and depressive symptoms provides two conceptualizations. The first conceptualization implies that loneliness, social anxiety, and depressive symptoms represent three separate, but related, latent constructs. This assumed distinctiveness is reflected, for example, in the separate definitions for the different internalizing problems and in the extensive but separate research lines devoted to each type of internalizing problem. A second conceptualization implies that loneliness, social anxiety, and depressive symptoms are reflections of a single latent construct. Two types of supportive evidence for each of these conceptualizations can be found, that is, correlations among the three internalizing problems and correlations between each internalizing problem and other variables. The substantial correlations among self-report measures of loneliness, social anxiety, and depressive symptoms (Mahon et al. 2006) seem to suggest that they can be represented using a single latent variable. In addition, the notable overlap in the interpersonal behaviors associated with each type of internalizing problem, such as poor social skills, a heightened sensitivity or fear for potential social threat, and social withdrawal (Ollendick and Hirshfeld-Becker 2002; Spithoven et al. 2017; Wang et al. 2012) points into the same direction. Finally, the three types of internalizing problems are associated with the same risk factors, for example, peer rejection (Boivin et al. 1995; Platt et al. 2013; Su et al. 2016) and low self-esteem (Sowislo & Orth, 2013; Vanhalst et al. 2013; van Tuijl et al. 2014).

In contrast, the empirical finding that loneliness, social anxiety, and depressive symptoms are differentially related with various types of outcomes (Hutcherson and Epkins 2009; La Greca and Harrison 2005; Starr and Davila 2008) seems to support the notion of three distinct but related latent variables. Social anxiety, compared to depressive symptoms, is more strongly associated with peer variables when controlled for depressive symptoms (Starr and Davila 2008). Depressive symptoms, by contrast, are more strongly associated with family variables than social anxiety when controlled for social anxiety (Starr and Davila 2008). Loneliness is not related to neuroticism and suicidal ideation when controlled for depressive symptoms. However, depressive symptoms are related to these outcome measures when controlled for loneliness (Lasgaard et al. 2011). Although these differential associations with outcomes suggest that loneliness, social anxiety, and depressive symptoms represent different types of internalizing problems, these studies do not directly address this issue. Factor analytic studies are needed to assess the distinctiveness of these constructs. Earlier empirical efforts that explicitly examined whether loneliness, social anxiety symptoms, and depressive symptoms are indeed three separate constructs using factor analysis are limited and focus on the distinction of just two types of internalizing problems, that is, loneliness and depressive symptoms (Cacioppo et al. 2006), social anxiety and depression (Gibb et al. 2005), or social anxiety symptoms and loneliness (Junttila et al. 2010). Only the latter study used an adolescent sample. These studies indicated that the internalizing problems can be validly discriminated pairwise. A recent study examined whether a three-factor model could be identified using well-established measures of loneliness, social anxiety symptoms, and depressive symptoms in a sample of college students. This study revealed that loneliness, social anxiety, and depressive symptoms as measured by the Revised UCLA-Loneliness Scale (UCLA-LS; Russell et al. 1980), the Social Interaction Anxiety Scale (SIAS; Mattick and Clarke 1998), and the Depression subscale of the Depression Anxiety Stress Scale (DASS-D; Lovibond and Lovibond 1995) can effectively be regarded as independent latent constructs (Fung et al. 2017). No similar study has been conducted yet in adolescence using age-appropriate measures.

The current study investigated whether loneliness, social anxiety, and depressive symptoms can be distinguished as three separate constructs in adolescence using confirmatory factor analysis in three different samples. Such analyses can have important implications for future research. If the results of the analyses suggest that the three-factor model best fits the data, the tacit assumption of distinctiveness holds. Future studies can then continue to focus on each of these problems separately and try to unravel the unique features of adolescent loneliness, social anxiety, and depressive symptoms. By contrast, if no clear three-factor solution can be found due to multiple cross-loadings, this substantial overlap might be due to a conceptual or a methodological problem or both. Specifically, on a conceptual level, the substantial overlap might suggest that there is no need to keep the research lines on loneliness, social anxiety, and depressive symptoms separated as is currently the case. Future research should then adopt a more integrative approach (Fung et al. 2017). Methodologically, the substantial overlap might suggest the need for a better operationalization of the three internalizing problems in case all three problems are examined simultaneously. However, given the results of prior research in college students, we expected that loneliness, social anxiety, and depressive symptoms would be readily distinguished from one another in adolescence as well.

Method

Participants

Three independent samples of adolescents from different schools in Flanders, the Dutch-speaking part of Belgium were used. In each sample, most adolescents attended the academic track.

Sample 1

The data for the first sample were derived from the PALS study (i.e., Personality and Loneliness/Solitude), a longitudinal study on the development of psychosocial well-being, personality, and identity throughout mid- and late adolescence, which was initiated in February 2010 (for details, see Teppers et al. 2013). For the current study, adolescents who attended Grade 9 and 10 at Time 1 (T1) were selected and their data from T1 to T3 was used. The total analytical sample consisted of 549 adolescents (Mage = 14.82 years, SD = 0.79), of whom 62.66% were girls. The majority of the adolescents lived with both their parents (81.6%).

Sample 2

The data for the second sample were derived from the STRATEGIES study (i.e., Studying Transactions in Adolescence: Testing Genes, Interactions, and Environments), a longitudinal study on the development of problem behavior in adolescence, which was initiated in February 2012 (for details see Janssens et al. 2015). A total number of 1116 adolescents participated in the study of which 6 participants were omitted from the current analyses because they did not fill out the questionnaires in a reliable way. Therefore, the final analytical sample comprised 1110 adolescents (Mage = 13.79 years, SD = 0.94), of whom 49.01% were girls. Adolescents attended Grade 7, 8, or 9 at Time 1 (T1) and their data from T1 to T3 was used. The majority of the adolescents were born in Belgium (94.61%) and lived with both their biological parents (79.2%).

Sample 3

The data for the third sample were derived from the EDA study (i.e., Emotional Development in Adolescence), a longitudinal study on adolescents’ emotional development, which was initiated in February 2013 (for details see Bastin et al. 2016; Nelis et al.2016). For the current study, adolescents who attended Grade 7, 8, and 9 at Time 2 (T2) were selected and their data from T2 to T4 was used. The total sample consisted of 1,423 adolescents (Mage = 13.58 years, SD = 0.96), of whom 52.74% were girls. The majority of the adolescents had the Belgian nationality (93.39%).

Procedure

Permission for the studies was obtained from the Institutional Review Board (IRB) at the researchers’ university (ML7972 and S54020 for Sample 2 and 3, respectively). Prior to data collection, in Samples 1 and 2, both adolescents and their parents gave active consent. In Sample 3, adolescents and their parents gave active and passive consent, respectively. Participants filled out paper-and-pencil questionnaires in their classroom during regular school hours. All measures were administered in Dutch, the native language of the participants. A research assistant was present during the test sessions to answer questions and to emphasize the voluntary and anonymous character of participation. Adolescents were informed that they could discontinue their participation at any time.

Measures

Loneliness

In all samples, the peer-related loneliness subscale of the Loneliness and Aloneness Scale for Children and Adolescents (LACA; Goossens 2016) was used to assess feelings of loneliness. This subscale consists of 12 items (e.g., “I feel sad because I have no friends” and “I feel left out by my friends”), which are answered on a 4-point Likert scale ranging from 1 (never) to 4 (often). A total score was computed as the mean of the 12 items, with higher scores representing higher levels of loneliness. The peer-related loneliness subscale shows good validity and reliability in terms of internal consistency in samples of Dutch-speaking adolescents (Goossens et al. 2015). In the present study, Cronbach’s alphas were high in all three samples (i.e., .86–.91).

Social anxiety

A Dutch translation of the Social Anxiety Scale for Adolescents (SAS-A; La Greca and Lopez 1998) was used to assess symptoms of social anxiety. In Samples 2 and 3 adolescents filled out the 12-item short version of the SAS-A (see Nelemans et al. 2017) and in Sample 1 the original 18-item version was used (e.g., “I worry about what others say about me” and “I feel shy with people I don’t know”). Items are rated on a 5-point Likert scale ranging from 1 (not at all) to 5 (all the time). A total score was computed as the mean of the 12 or 18 items, with higher scores reflecting higher levels of social anxiety. Previous studies have indicated that both the original and short version of the SAS-A show good reliability in terms of internal consistency in an English speaking adolescent sample (La Greca and Lopez 1998) and in two Dutch-speaking samples, that is, Sample 1 and 2 of the current study (Nelemans et al. 2017), respectively. In the present study, Cronbach’s alpha’s were high in all three samples (i.e., .91–.92).

Depressive symptoms

Depressive symptoms were assessed using the 20-item (Hooge et al. 2000; Radloff 1977) and the 12-item shortened version (Roberts and Sobhan 1992; Bouma et al. 1995) of the Center for Epidemiologic Studies Depression Scale (CES-D; Radloff 1977) in Samples 1 and 2, respectively, and the Child Depression Inventory (CDI; Kovacs 2003; Timbremont and Braet 2002) in Sample 3.

The CES-D is intended to measure depressed mood in the general population (e.g., “During the past week I enjoyed life”, reverse coded, and “During the past week my sleep was restless”). Items are rated on a 4-point Likert scale ranging from 0 (rarely or never) to 3 (mostly or always). A total score was computed as the mean of the 12 or 20 items, with higher scores reflecting higher levels of depressive symptoms. Previous research has shown that the full version of the CES-D shows good reliability in terms of internal consistency in samples of Dutch-speaking adults (Hooge et al. 2000; Wu et al. 2016) and in English speaking adolescents (Radloff 1977; Siddaway et al. 2017). Acceptable internal consistency for the 12-item short version of the CES-D has been found in an English speaking sample of adolescents (Poulin et al. 2005). In the present study, Cronbach’s alphas for both versions of the CES-D were high in Sample 1 and 2 (i.e., .82 and.93).

The CDI consists of 27 items that measure cognitive, affective, and behavioral symptoms of depression during the past two weeks (e.g., “Nothing is fun at all”, and “I am sad all the time”). For each item, adolescents chose one out of three statements describing different levels of symptom severity. Items are rated on a scale ranging from 0 to 2. Higher mean values indicate greater symptom severity. Both the original version and the Dutch adaptation have shown to be reliable in terms of internal consistency and valid in samples including adolescents (Kovacs 2003; Timbremont and Braet 2002). In the present study, Cronbach’s alpha was high in Sample 3 (i.e., .87).

Statistical Analysis

To investigate whether loneliness, social anxiety, and depressive symptoms are distinct constructs, confirmatory factor analyses were conducted in all three samples using Mplus Version 8 (Muthén and Muthén 1998–2017). Items belonging to a certain measure intended to tap into one of the three internalizing problems, were specified to load on that specific construct. To avoid bias due to item overlap between the loneliness and depressive symptoms measures, in Sample 2 the item “During the past week, I felt lonely” of the CES-D and in Sample 3 the items “I feel lonely all the time” and “I do not have any friends” of the CDI were not included in the analyses. Because the item scores were treated as ordered categorically, the robust weighted least squares estimator was applied (Muthén 1984). In order to estimate all factor loadings and intercepts, we constrained the latent means to zero and the latent variances to one (Van de Schoot et al. 2012). Factor loadings were evaluated whereby loadings above .32 were considered as substantial (Tabachnick and Fidell 2001).

First, we examined whether the three-factor model was empirically supported. We evaluated the model fit of the three-factor model by means of several fit indices. The Chi-square statistic was evaluated with a non-significant chi-square being indicative of good model fit. However, because Chi-square statistics have been found to be highly sensitive to sample size (Barrett 2007), other fit indices were taken into account as well. Specifically, we also relied on the comparative fit index (CFI), the Tucker-Lewis Index (TLI), and the root mean square error of approximation (RMSEA). A model is regarded as fitting the data well when the CFI and TLI exceed .95, and when the RMSEA is lower than .06 (Hu and Bentler 1999). An adequate fit is achieved when the CFI and the TLI exceed .90, and when the RMSEA is lower than .08 (Kline 2005).

Second, we tested whether the three-factor model would provide a superior fit compared to alternative, simpler models. First, we compared the three-factor model with the simplest model including just a single factor. Next, the three-factor model was compared to three two-factor models. Specifically, in each of the two-factor solutions the items from two of the three scales were specified to load onto one factor and the items from the remaining third scale were specified to load onto the other factor. To compare the different confirmatory factor models to one another, we relied on the guidelines of Chen (2007). A significant difference in model fit between the three-factor model and both the two-factor models and one-factor model indicates that the unconstrained, three-factor model can be retained. This is the case when the p value for Δχ2 is below .05, ΔCFI exceeds .01 supplemented by ΔRMSEA exceeding .015 (Chen 2007).

Results

Fit indices for the three-factor models in all three samples are presented in Table 1. Following the rules of thumb of Hu and Bentler (1999), in all three samples, the three-factor model including loneliness, social anxiety, and depressive symptoms as separate factors showed an adequate fit to the data in most cases. The CFI revealed adequate fit in both Samples 2 and 3 (following rounding). With values of .89 and .88, the CFI and TLI, respectively, did not reach the desirable cutoff of .90 in Sample 1. The RMSEA, in turn, showed that the three-factor model fitted the data adequately in all three samples.

Table 1 Fit indices for the various models in the three samples
Full size table

Standardized factor loadings for the final three-factor solution in all three samples are presented in Tables 2 to 4. All factor loadings were significant at p< .001 and substantial (i.e., > .32; Tabachnick and Fidell 2001), except the factor loadings of Item 2 of the CES-D in Sample 1 (i.e., “I had trouble keeping my mind on what I was doing”) and Item 26 of the CDI (i.e., “I never do what I am told”) in Sample 3.

Table 2 Standardized factor loadings for the modified three-factor model comprising loneliness, social anxiety, and depressive symptoms (Confirmatory Factor Analysis - Sample 1)
Full size table
Table 3 Standardized factor loadings for the three-factor model comprising loneliness, social anxiety, and depressive symptoms (Confirmatory Factor Analysis - Sample 2)
Full size table
Table 4 Standardized factor loadings for the modified three-factor model comprising loneliness, social anxiety, and depressive symptoms (Confirmatory Factor Analysis - Sample 3)
Full size table

In the final models of all three samples, the factors were moderately to strongly correlated. The correlation between the loneliness factor and the social anxiety factor was r = .67, .58, and .59 in Samples, 1, 2, and 3, respectively, similarly the correlation between the loneliness factor and depressive symptoms factor was r = .48, .56, and .52 in Samples 1, 2, and 3, respectively. Finally the correlation between the social anxiety factor and the depressive symptoms factor was r = .56, .49, and .48 in Samples 1, 2, and 3, respectively.

The alternative, simpler models, that is, a model comprising a single factor and the different two-factor models, did not show good fit to the data in any of the three samples (Table 1). In addition, in line with the guidelines of Chen (2007), the three-factor model showed a superior fit compared to the different two-variable models and the single factor model in Sample 1 (i.e., Δχ2s < 1894.456, ps < .001, ΔCFI > .04, and ΔRMSEA > .015, except for two comparisons where the difference was .014), Sample 2 (i.e., Δχ2s < 8476.08, ps < .001, ΔCFI > .01, and ΔRMSEA > .015) and Sample 3 (i.e., Δχ2s < 8729.38, ps < .001, ΔCFI > .07, and ΔRMSEA > .015).

Discussion

The aim of the present study was to examine whether adolescent loneliness, social anxiety, and depressive symptoms could be regarded as three distinguishable internalizing problems in adolescence. To address this aim, confirmatory factor analyses were conducted in three large samples of adolescents, which allowed us to test for the robustness of findings across samples. In all three samples, a relatively clear distinction could be made among loneliness, social anxiety, and depressive symptoms as measured by the LACA, SAS-A, and CES-D or CDI, respectively. In other words, we found empirical support for the tacit assumption that loneliness, social anxiety, and depressive symptoms represent distinct constructs. This result confirms an earlier factor analysis including all three internalizing problems (Fung et al. 2017) and some factor analyses including two instead of all three internalizing problems (Cacioppo et al. 2006; Junttila et al. 2010). However, these studies were mainly conducted in adults and college students rather than adolescents. In addition, the measures used to assess loneliness, social anxiety, and depressive symptoms in the present study are commonly used in research on adolescents. As such, they are different from the instruments used in research with adults. The current study, therefore, represents a valuable contribution to the extant literature by providing crucial evidence for the distinctiveness of loneliness, social anxiety, and depressive symptoms in adolescence.

Based on these results, the current tradition of separate research lines for the three internalizing problems seems justified (Fung et al. 2017). However, an important next step for future research would be to unravel the key components that account for the distinctiveness among loneliness, social anxiety, and depressive symptoms. In addition, it remains unclear whether these internalizing problems show similarities in their etiology and how they relate to each other over time. Examining how these internalizing problems develop conjointly might result in a deeper understanding of the development of adolescents’ internalizing problems. For example, investigating the temporal sequence among loneliness, social anxiety, and depressive symptoms using cross-lagged analysis might provide more insight into whether internalizing problems potentially function as risk factors for one another across time. Because the results of the current study indicate that loneliness, social anxiety, and depressive symptoms represent distinct entities, an important prerequisite for cross-lagged analyses or other complex statistical analyses that examine these internalizing problems’ developmental interplay is fulfilled. Consequently, these analyses can now be validly conducted.

Strengths and Limitations

The present study has several strengths, such as the use of three large independent samples of adolescents and the use of two established and validated measures of depressive symptoms to test for the robustness of findings across measures. However, the results of the present study should be interpreted in the light of some limitations. First, to assess feelings of loneliness and social anxiety, highly similar measures have been used across the different samples. Although this approach allowed us to test the robustness of the findings across samples, a drawback is that the results of our study regarding the distinctiveness of loneliness, social anxiety, and depressive symptoms cannot be generalized to other measures than the ones used in the current study. Second, we examined the distinctiveness of three internalizing problems in three community samples of adolescents, which were all living in the Dutch-speaking part of Belgium. Therefore, care should be taken when generalizing the findings of the current study to adolescents living in other parts of the world.