Introduction

Obsessive–compulsive disorder (OCD) is characterized by the presence of obsessions and/or compulsions that produce distress and disability [1]. At least 50 % of obsessive–compulsive symptoms begin during childhood [2]; as many as 2 % of youth meet diagnostic criteria for OCD [3], with up to 19 % presenting subclinical symptoms [4–6].

Given the incidence of the disorder and its associated deleterious consequences [7], instruments have been developed to assess obsessive–compulsive symptomatology in children and adolescents [8]. International mental health organizations such as the National Institute for Health and Clinical Excellence (NICE) [9] and the American Academy of Child and Adolescent Psychiatry [3] recommend routine screening of young people at risk, either in general practice or in other settings where they may present for help. This screening requires short, easy-to-use, sensitive, inexpensive, and widely available measures. Clinical trials, especially those documenting real-world efficacy, also require brief, focused measures that can detect changes in core symptomatology.

The existence of well-validated instruments to assess childhood obsessive–compulsive symptoms is critically important for the development and dissemination of effective interventions, as well as for initiatives aimed at early detection and treatment. The evidence-based assessment (EBA) classification [10] defines the evidence base of a measure according to three levels of empirical support: (1) well-established assessment (i.e., reliability and validity have been demonstrated in at least two published studies by two research teams); (2) approaching well-established assessment (i.e., reliability and validity have been demonstrated in at least two published studies by one research team; or two research teams published studies offering mixed psychometric results); and (3) promising assessment (i.e., reliability and validity have been demonstrated in at least one published study).

Currently, the clinician-administered Children’s Yale-Brown Obsessive–Compulsive Scale (CY-BOCS; [11]) is the gold standard for the assessment of OCD symptom presence and severity given its excellent psychometric properties and treatment sensitivity [12]. However, the clinician-administered format is lengthy, requires extensive training, and is not well suited to community settings. The self-report format [13, 14] is also too long for screening purposes outside clinical settings, and certain questions (e.g., resistance) may be difficult for youth to understand.

There are six child-report instruments designed to assess pediatric OCD symptoms and severity that have received empirical support. Among these, the Obsessive Compulsive Inventory-Child Version (OCI-CV) [15] is a well-established assessment tool that assesses the frequency of, and distress associated with, obsessive–compulsive symptoms. The other five instruments can be regarded as approaching well-established or promising assessment tools: examples are the Children’s Obsessional Compulsive Inventory (CHOCI) [16], the Short OCD Screener (SOCS) [9, 17], and the Leyton Obsessional Inventory Child-Version (LOI-CV) [18], although the psychometric properties of the LOI-CV are poor [12]. All of these tools, however, have limitations for screening purposes, whether due to their length, their ease of use, or availability (for further details, see [12]).

The Children’s Florida Obsessive–Compulsive Inventory (C-FOCI) [19] was developed in an attempt to remedy this situation. The C-FOCI is a brief self-report questionnaire that assesses the presence of obsessive–compulsive symptoms and their associated severity in a manner that parallels the CY-BOCS. What distinguishes the C-FOCI, however, is that it is easy to comprehend, provides a unitary severity scale, and removes the “resistance against symptoms” item which many patients have difficulty understanding. It was developed for its use as a screening instrument for community and general clinical populations. A primary difference from other pediatric OCD self-report measures is that the C-FOCI items are divided into two parts: the symptom checklist and the Severity Scale (see [19] for a further description of the C-FOCI development). The symptom checklist enquires about the presence of 17 obsessions and compulsions that have been endorsed with relative frequency among youth with OCD. The Severity Scale is a unitary scale of five items that collectively assess the severity of all obsessive–compulsive symptoms without artificially dividing them into obsessions and compulsions.

The psychometric properties of the C-FOCI have been validated in one publication involving two samples (clinical and community groups) [19]. Results for the clinical sample (n = 82 OCD participants aged 7–20 years) showed that internal consistency was adequate for both the Severity Scale (α = .79) and the symptom checklist (KR-20 = .76). The most frequent symptoms, endorsed by over 50 % of the clinical sample, were: contamination concerns; ritualized handwashing, cleaning, or grooming; symmetry and ordering compulsions; and reassurance seeking and confessing. Convergent validity was demonstrated through significant relationships with the CY-BOCS (r = .50) and Child Obsessive–Compulsive Impact Scale-Child and Parent Versions (COIS-C/P; r = .42–.49) [20]. In addition, C-FOCI symptom dimensions were significantly related to the corresponding CY-BOCS symptom domains (r = .25–.33). The C-FOCI Severity Scale and symptom checklist correlated significantly and moderately with several measures of depressive and anxiety symptoms (Multidimensional Anxiety Scale for Children, MASC [21]; Children’s Depression Inventory, CDI [22]; and the Internalizing score of the Child Behavior Checklist, CBCL [23]; r = .40–.48, and r = .35–.61, respectively), although not with the Externalizing score of the Child Behavior Checklist (r = .11 and r = .13, respectively). Finally, the C-FOCI was sensitive to changes after cognitive-behavioral treatment [19].

Despite the strengths of the C-FOCI, several psychometric issues have yet to be addressed, including the investigation of its factor structure and its invariance, test–retest reliability estimations, the lack of other evidence of convergent–divergent validity, and specificity and sensitivity data. The importance of sensitivity and specificity data analysis for screening purposes is worth mentioning here; specificity and sensitivity are key indices of a measure’s value for screening children who may require a more comprehensive assessment for OCD.

Consequently, the main objective of the present study was to extend the validation of the C-FOCI with the participation of a second research group who carried out an empirical examination of the tool’s utility for assessing obsessive–compulsive symptoms across a wide age range in clinical and community samples of children and adolescents, as well as to provide for a psychometrically sound measure of obsessive–compulsive symptoms for use in Spanish youth. There were seven research questions: (1) What symptoms are frequently endorsed by clinical and community samples?; (2) Which factor structure best reflects the construct(s) measured by the scale, and do respondents across clinical and community groups attribute the same meaning to the latent constructs under study (metric invariance); (3) What is the internal consistency and temporal stability of the C-FOCI?; (4) Is the C-FOCI useful for discriminating between clinical and community groups?; (5) Does the C-FOCI Severity Scale correlate with measures of obsessive–compulsive symptoms and severity, and of anxiety and depressive symptomatology?; (6) Is the C-FOCI a diagnostically accurate measure, in terms of sensitivity and specificity, for detecting pediatric OCD?; and (7) Is the C-FOCI a good measure for predicting the presence of pediatric OCD? We hypothesized that C-FOCI scores would show: (1) different OCD manifestations according to clinical and community samples; (2) the metric invariance of a bi-factor structure across samples; (3) high reliability in terms of internal consistency and temporal stability; (4) known-group validity; (5) concurrent validity, with a higher correlation of C-FOCI scores with OCD symptoms than with anxiety and depressive measures; (6) acceptable accuracy for screening purposes; and (7) acceptable predictive validity to identify the presence of childhood OCD.

Methods

Participants

The clinical sample consisted of 94 (female = 48) child and adolescent patients who had a current primary diagnosis of OCD according to DSM-IV-TR criteria [24]. Their mean age was 14.62 years (SD = 2.65; range = 8–19 years). All clinical participants were recruited from the Child and Adolescent Psychiatry and Psychology Unit of the Hospital Clinic in Barcelona and the Child and Adolescent Clinical Psychology Unit of the University Miguel Hernández of Elche. Axis I diagnoses were made by trained professionals (a psychologist and psychiatrist) using the validated Spanish version [25] of the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL) [26], with both parents and the child as informants. Exclusion criteria included presence of a psychotic disorder, autism spectrum disorder, or intelligence quotient (IQ) <70. Although all participants had a primary diagnosis of OCD, the presence of other non-primary comorbid Axis I disorders was permissible given the high rates of comorbidity seen in childhood OCD [27]. Forty-five children (47.9 %) did not present any comorbidity; among the remaining 49 (52.1 %) participants, 35 (37.2 %) presented with just one comorbid condition, nine (9.6 %) with two, three (3.2 %) with three, and two (2.1 %) with four associated disorders. Specifically, we found the following rates of associated secondary disorders: generalized anxiety disorder 21.3 % (n = 20), eating disorders 9.6 % (n = 9), Tourette’s disorder 8.5 % (n = 8), attention deficit/hyperactivity disorder 7.4 % (n = 7), major depression 7.4 % (n = 7), social phobia 4.2 % (n = 4), specific phobia 3.2 % (n = 3), oppositional defiant disorder 3.2 % (n = 3), dysthymia 2.1 % (n = 2), and bipolar disorder 2.1 % (n = 2).

The final community sample consisted of 1068 students (female = 542) from five schools in the Levante region of eastern Spain. Their mean age was 12.38 years (SD = 2.69; range = 8–18 years). The participating centers were located in towns and cities of varying size in both rural and urban areas; three of them were state centers and two were semi-private. Consequently, the socio-economic status of the sample ranged widely and was representative of the recruitment community (n = 1174). The inclusion criteria for this sample were the following: (1) registration on the official school lists; (2) provision of written informed consent; (3) attending school during the assessment; and (4) the ability to understand and complete the survey. Finally, participants who did not perform the measurements at either time point were excluded, as were those who did not respond to some of the questionnaires. In all, 106 participants were excluded, representing an attrition rate of 9 %. There was no diagnostic assessment of psychiatric disorders in the community sample, apart from the administration of some self-report questionnaires referring to anxiety and depressive symptoms.

Measures

Children’s Florida Obsessive–Compulsive Inventory (C-FOCI)

Based on the Florida Obsessive–Compulsive Scale [28], the C-FOCI [19] was developed to assess the presence and severity of obsessive–compulsive symptoms. Its nature and psychometric properties have been reviewed above. The original scale was translated and adapted into Spanish following international guidelines and technical standards [29], as follows: (1) the Spanish translation was authorized by Eric Storch and Wayne Goodman in 2010; (2) a translation/back-translation procedure was conducted; (3) expert judgement to support the linguistic equivalence and understanding of the items was provided by specialists in related fields (child and adolescent clinical psychology and psychiatry; psychological assessment; psychometrics, etc.); (4) experts evaluated the correspondence of each item with the belonging dimension in order to support the content validity of construct; (5) the Spanish version of the C-FOCI was then administered to a pilot sample of children aged 8–10 years, balanced in terms of gender distribution (50/50), so as to ensure that the item content could be understood by young children. For this step we used the cognitive interview technique, a qualitative methodology.

Obsessive Compulsive Inventory-Child Version (OCI-CV)

The Spanish version of the OCI-CV consists of a childhood self-report of 21 items scored on a 3-point Likert scale (0 = never, 2 = always). Seven scores are derived: doubting/checking, obsessing, hoarding, washing, ordering, neutralizing, and a total score. Both the English [15] and Spanish versions [30, 31] of the OCI-CV show excellent psychometric properties. Internal consistency in our sample ranged from .78 to .91 for clinical sample and from .62 to .90 for community sample.

Short Obsessive–Compulsive Disorder Screener (SOCS)

The SOCS [17] is a childhood self-report of seven items that assess the presence of common obsessions and compulsions. This instrument showed good internal consistency, test–retest stability, a unidimensional factor structure, and excellent sensitivity to detect OCD among both English [17] and Spanish [32] children and adolescents. Internal consistency values were .79 and .71 for clinical and community samples respectively.

Revised Child Anxiety and Depression Scale (RCADS-30)

The RCADS-30 [33] is an abbreviated 30-item version of the RCADS [34], a childhood self-report questionnaire assessing anxiety and depressive symptoms in children and adolescents. Symptoms of the following conditions are evaluated on the RCADS-30: panic disorder, social phobia, separation anxiety disorder, generalized anxiety disorder, OCD, and major depressive disorder. Scale items are scored from 0 to 3, corresponding to “never”, “sometimes”, “often” and “always”. The Spanish version of the RCADS-30 has shown excellent psychometric properties that are equivalent to those of the full version [33]. Internal consistency values ranged from .73 to .93 for the clinical sample and from .70 to .91 for the community sample.

Children’s Yale-Brown Obsessive Compulsive Scale (CY-BOCS)

The CY-BOCS is a clinician-administered, semi-structured interview for assessing the severity of childhood obsessive–compulsive symptoms [11]. The scale assesses the content of the patient’s obsessions and compulsions. These symptoms are then rated on a 5-point scale covering five areas each for obsessions and compulsions: time spent daily on the symptoms, interference caused by the symptoms, level of distress, degree of resistance, and control. An individual severity score is derived for both obsessions and compulsions, while the Total Severity score, ranging from 0 to 40, is a sum of all items, with a higher score indicating greater severity.

Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL)

The K-SADS-PL diagnostic interview is useful for the cross-sectional and longitudinal evaluation of psychopathology in children and adolescents [25]. It has shown good to excellent reliability for the majority of mental disorders assessed (k = .76–1). Specifically, its reliability for any anxiety disorder was k = .84 [25]. In the present study, 20 % of the interviews were video-taped for inter-rater agreement and then observed by two independent judges. The inter-rater reliability for the diagnosis of OCD was excellent (k = .90), whereas the reliability for the diagnosis of comorbid disorders ranged from .70 to .90.

Procedure

The study was approved by the research and ethics committees of the University Miguel Hernández of Elche and the Hospital Clinic of Barcelona. Trained professionals (a psychologist and psychiatrist, who were native Spanish speakers) administered the C-FOCI and the other self-report questionnaires in both the clinical and community samples, as well as the K-SADS-PL and CY-BOCS to the clinical sample alone, at the time of study entry. Patients were contacted 4 weeks after the first evaluation, at which point the children once again completed the same questionnaires in person.

Written informed consent for participation in the study was obtained from parents and/or guardians and from children and adolescents in both clinical and community settings. Furthermore, all schools and clinical centers participated voluntarily in this study, after obtaining permission from the school principal and the educational psychology service in the case of the schools, or from the managing directors in the case of the clinical centers.

Data Analysis

Different sets of analyses were conducted in accordance with the dichotomous (symptom checklist) or ordinal (Severity Scale) nature of the variables (not ratio scales). Thus, we selected the most appropriate statistical tests for each set. First, descriptive statistics were computed. We calculated the frequency counts for each of the C-FOCI symptom checklist items and the differences between clinical and community groups on Severity Scale items using the Mann–Whitney U test. The Probability of Superiority (PS) measure [35] was used to estimate the effect size of any differences; PS is a non-parametric effect size index, is robust against unequal sample sizes, and provides an alternative to Cohen’s d when data violate parametric assumptions. Independent t tests were used to examine gender, age, and comorbid conditions differences in the C-FOCI symptom checklist and Severity Scale scores within each sample. Age was divided into two groups consisting of children aged 8–12 and adolescents 13–19 years in the clinical sample, and children 8–12 and adolescents 13–18 years in the community sample.

The next stage involved examining the factor structure and the metric invariance of the C-FOCI across both samples. Following Storch et al. [19], the hypothesis that all items are grouped into two independent factors was tested by confirmatory factor analysis for both the clinical and community samples as a whole. We also tested whether the C-FOCI presents metric invariance. This was performed once, not systematically (i.e., we did not begin by checking the configuration invariance; later the invariance of the factor loadings; then the measurement errors, and finally the intercepts). Across cases, all the restrictions were introduced simultaneously, checking the metric invariance between the groups compared with a single calculation. Polychoric correlation matrices and the diagonally weighted least squares method were used in all cases (LISREL 8.8, DWLS procedure). Lastly, the following were used as goodness of fit indices [36]: RMSEA equal to or less than .08; CFI and NNFI (or TLI) superior to .95; and the Satorra–Bentler Chi square. The CFI was also used to compare the basic model (total sample) with the metric invariance model between the clinical and community samples. According to Cheung and Rensvold [37], the invariance between samples is admissible when the difference of CFIs (ΔCFI) is less than or equal to .01. These authors also assert that ΔCFI is a better estimator of invariance admissibility than Δχ2.

In the third stage of the analysis, we provided reliability estimates by examining internal consistency and 4-week temporal stability for both samples. Internal consistency was estimated by calculating the Kuder–Richardson-20 (KR-20) for the symptom checklist using SPSS v22, while for the Severity Scale it was assessed with standardized alpha using FACTOR 10.3 software. KR-20 is a measure of internal consistency for measures with dichotomous choices, and it is analogous to Cronbach’s α. We calculated the intraclass correlation coefficient (ICC) to assess the test–retest association over a 4-week interval, as well as a paired sample t test between Time 1 and Time 2 for 90 participants from the clinical sample and 603 from the community sample. The average time between the two administrations was 28 days. We then examined known-group validity using the Mann–Whitney U test and the Probability of Superiority (PS) measure [35] to estimate the effect size of any differences between the group with OCD and the group without OCD on the symptom checklist and the Severity Scale. The probabilities of superiority corresponding to Cohen’s (1988) standards for small (r = 0.10, d = 0.20), medium (r = .30, d = 0.50), and large (r = .50, d = 0.80) effect sizes are approximately 0.56, 0.64, and 0.71 respectively [38, 39]. Next, the concurrent and divergent validity of the C-FOCI was examined on the basis of correlations between the C-FOCI and other empirically validated measures of OCD, specific dimensions of OCD, and related variables such as anxiety or depressive symptoms. For these analyses we calculated Pearson correlation coefficients, using Cohen’s criteria as a measure of effect size (ES): small (.10), medium (.30), and large (≥.50) [38, 39].

The diagnostic accuracy of the C-FOCI for correct classification of participants with or without OCD was determined by studying the ROC curve as well as providing sensitivity and specificity rates. We followed the traditional academic point system described by Metz [40] for classifying the accuracy of a diagnostic measure focused on ROC curve: .90–1.00 = excellent, .80 to .90 = good, .70 to .80 = fair, .60 to .70 = poor, <.60 = fail.

Finally, the predictive validity of the C-FOCI with respect to a K-SADS-PL diagnosis of OCD was tested through binary logistic regression analyses. The outcome variables were the possible diagnoses according to the K-SADS-PL (i.e., OCD or non-OCD), while the predictor variables were scores on the symptom checklist and the Severity Scale.

Results

Descriptive Statistics

For the clinical sample the number of symptoms endorsed ranged from 0 to 13 (M = 6.46, SD = 3.50). The mean score for the Severity Scale was 8.47 (SD = 5.16). Boys (n = 45; M = 6.04, SD = 3.72) and girls (n = 47; M = 6.77, SD = 3.34) endorsed a similar number of symptoms on the symptom checklist (t(90) = −0.98, p = .33), and girls showed a similar symptom severity to boys on the Severity Scale (M = 7.82, SD = 4.96 vs. M = 9.04, SD = 5.41 respectively; t(90) = −1.12, p = .26). A similar number of symptoms were reported by children (8–12, n = 23) and adolescents (13–19, n = 67) on the symptom checklist (M = 5.91, SD = 3.72 vs. M = 6.55, SD = 3.35 respectively; t(88) = −0.74; p = .46), and these two groups were also comparable in terms of symptom severity (M = 7.70, SD = 4.79 vs. M = 8.75, SD = 5.42; t(88) = −0.82, p = .41). The Symptom and Severity Scales were not differentially scored by children with no comorbid conditions versus those who had one or more comorbid conditions (symptom checklist: M = 6.18, SD = 3.64 vs. M = 6.85, SD = 3.31; t(92) = −0.90, p = .37; Severity Scale: M = 7.85, SD = 5.31 vs. M = 9.33, SD = 4.87; t(92) = −1.38, p = .17 respectively).

For the community sample, the number of symptoms endorsed ranged from 0 to 17 (M = 5.95, SD = 3.77). The mean for the Severity Scale was 4.69 (SD = 3.77). Boys and girls endorsed a similar number of symptoms on the symptom checklist (5.86 vs. 6.03), and they showed comparable symptom severity on the Severity Scale (4.65 vs. 4.75; t(1064) = −0.44, p = .66 and t(1064) = −0.75, p = .45 respectively). A greater number of symptoms were reported on the symptom checklist by children (8–12, n = 554) than by adolescents (13–18, n = 514) (t(1066) = 10.97, p < .001; 7.11 ± 3.67 vs. 4.71 ± 3.47). Children reported greater symptom severity than did adolescents on the Severity Scale (t(1066) = 2.23, p < .05; 4.94 ± 3.69 vs. 4.43 ± 3.83) (Table 1).

Table 1 Frequencies, means, and standard deviations for C-FOCI items
Full size table

Confirmatory Factor Analysis

As shown in Table 2, goodness of fit indices for the total sample indicated that this model fits the data reasonably well for two independent factors, the symptom checklist and the Severity Scale, and we found metric invariance (configurational, of factor loadings, of measurement errors, and of intercepts) between the clinical and the community group for the symptom checklist and the Severity Scale. The RMSEA was below .08, and the CFI and NNFI were above .90 in all cases; ΔCFI was equal to 0.

Table 2 Confirmatory factor analysis (total sample: n = 1162) and multi-group confirmatory factor analyses (diagonally weighted least squares; polychoric correlation matrix) for children from a clinical setting (n = 94) and children from the community (n = 1068)
Full size table

Table 3 shows the degree of relationship (standardized lambda weights) for each item with respect to the symptom checklist and the Severity Scale. All item weights were above .44 for the symptom checklist and above .70 for the Severity Scale.

Table 3 Item content, item factor loading (lambda)
Full size table

Reliability

The internal consistency (KR-20) for the symptom checklist was .73 and .79 for the clinical and community samples respectively. For the Severity Scale the internal consistency (standardized alpha) was .89 and .82 for the clinical and community groups respectively.

Regarding 4-week test–retest reliability, in the clinical sample the two mean scores for the symptom checklist were 6.30 (SD = 3.46) and 5.56 (SD = 3.34), with ICC = .85; the corresponding values for the Severity Scale were 8.30 (SD = 5.18) and 8.06 (SD = 4.76), with ICC = .90. In the community sample the two mean scores for the symptom checklist were 6.31 (SD = 3.75) and 5.70 (SD = 3.77), with ICC = .83; the corresponding values for the Severity Scale were 4.84 (SD = 3.73) and 4.22 (SD = 3.78), with ICC = .74. Correlations were statistically significant (p < .01) in both groups.

In regards to the paired sample t tests between Time 1 and Time 2, in the clinical sample the paired sample t test showed statistically significant differences for the symptom checklist (t(89) = 2.88; p < .01), with a small effect size (d = 0.30). There were no significant differences for the Severity Scale (t(89) = 0.77; p > .05). In the community sample, the paired sample t test showed statistically significant differences for the symptom checklist (t(602) = 5.22; p < .001) and Severity Scale (t(602) = 4.43; p < .001), but in both cases the effect sizes were small (d = 0.21 and 0.18 respectively).

Known-Group Validity

Table 1 shows the means and standard deviations of the C-FOCI for the clinical and community samples. On the symptom checklist, and in comparison with the community sample, participants from the clinical setting endorsed a significantly higher percentage of items related to obsessions involving images of death or horrible things (3), ritualized handwashing, cleaning, or grooming (9), repetitive routine actions (12 and 14), needing to touch objects or people (13), and avoiding colors or names that are associated with scary events or thoughts (16). By contrast, the community sample showed a significantly higher percentage of endorsements for items related to being worried about “Accidentally hitting a pedestrian with your car or hurting someone” (5) and “Losing something valuable” (7) or needing to “Examine the body for signs of illness” (15) (p < .001). There were no statistically significant differences between the two groups in the total number of symptom checklist items endorsed. Regarding the Severity Scale, participants from the clinical setting obtained higher scores than did those from the community sample (PS = 0.71; a large effect size according to Grissom [41]), with this being the case for each item of the severity scale (PS = 18: 0.66; 19: 0.73; 20: 0.71; 21: 0.60; and 22: 0.64).

Convergent and Divergent Validity

The mean score of severity of the CY-BOCS was 27.34 (SD = 7.12; range = 12–40). In the clinical sample the C-FOCI symptom checklist correlated significantly with the CY-BOCS Severity Scale (r = .27), and its correlations with the OCI-CV subscales and total score, the SOCS, and the RCADS-30 OCD subscale achieved medium to large effect sizes (r = .39–.77). The C-FOCI Severity Scale was significantly related to the CY-BOCS Severity Scale (r = .40), and its correlations with the aforementioned OCD-related measures indicated medium to large effects (r = .38–.72). The exceptions were the OCI-CV Hoarding, Ordering and Neutralizing subscales, which were weakly related to the C-FOCI Severity Scale (r = .20–.29).

In the community sample, the symptom checklist was correlated with OCI-CV subscales and total score, the SOCS, and the RCADS-30 OCD subscale, showing medium to large effect sizes (r = .37–.71). For the Severity Scale, the effect sizes of the correlations with the same measures were between small and medium (r = .27–.54).

With regard to related construct measures such as anxiety and depression symptoms, all correlations for both C-FOCI subscales were significant with small-to-medium effect sizes for the clinical (r = .25–.62) and community samples (r = .27–.52). The only exception in both groups was the RCADS total score, which showed a large effect size in its correlation with the C-FOCI symptom checklist and Severity Scale (see Table 4).

Table 4 Convergent/divergent validity
Full size table

Sensitivity and Specificity of the C-FOCI

The area under the curve (AUC) for the C-FOCI symptom checklist was .63 (SE = .03, p < .001), suggesting a 63 % probability that a participant with OCD would score higher on the C-FOCI symptom checklist than a participant from the community sample. Table 5 shows the sensitivity and specificity for the set of items. Cut-off scores were selected to provide the best balance between sensitivity and specificity for each measure. Therefore, applying the Youden Index the symptom checklist cut-off score of 7 showed a sensitivity of 57 % and a specificity of 48 %, a Youden Index of .06, and accuracy or informedness (subjects correctly classified) of 51 %. Although a score of 6 is less specific (38 %), it is more sensitive (69 %), so it could be chosen as a good cut-off score to detect OCD.

Table 5 Sensitivity, specificity, Youden Index, and accuracy of the symptom checklist
Full size table

The AUC of the C-FOCI Severity Scale was .77 (SE = .02, p < .001), suggesting a 77 % probability that a youth with OCD would obtain a higher severity score than a young person from the community sample. Table 6 shows the sensitivity and specificity for the Severity Scale. Cut-off scores were selected to provide the best balance between sensitivity and specificity for each measure. Thus, a Severity Scale cut-off score of 7 showed sensitivity of 63 % and specificity of 63 %, a Youden Index of .26, and accuracy or informedness (subjects correctly classified) of 37 %. A score of 6 displays slightly better sensitivity (73 %) but lower specificity (53 %).

Table 6 Sensitivity, specificity, Youden Index, and accuracy of the Severity Scale
Full size table

Predictive Validity

Regarding the symptom checklist, the predictive model accounted for 8.4 % (Nagelkerke R 2) of the variance in OCD diagnoses, and resulted in 92 % classification accuracy. The symptom checklist was related to diagnostic status, Wald (1) = 42.36, p < .001, such that higher symptom checklist scores were better predictors of OCD diagnoses, OR = 1.55 (95 % CI = 1.36–1.78).

With respect to the Severity Scale, the predictive model accounted for 13 % (Nagelkerke R 2) of the variance in OCD diagnoses, and resulted in 92 % classification accuracy. The Severity Scale was related to diagnostic status, Wald (1) = 65.86, p < .001, such that higher Severity Scale scores were better predictors of OCD diagnoses, OR = 1.23 (95 % CI = 1.17–1.29).

Discussion

We report additional psychometric properties of the C-FOCI in a European (Spanish) sample. Although preliminary research had supported the psychometric properties of the instrument [19], several other issues remained to be addressed, including its factor structure and metric invariance across clinical and community samples, the limited evidence of convergent-divergent validity, test–retest reliability, and sensitivity/specificity data.

Findings regarding the incidence of obsessions and compulsions in the community sample indicated that young people without OCD experience “normal” obsessions that are comparable in content to those with OCD, and that what distinguishes people with “normal” obsessions from people with OCD is the severity of obsessions and compulsions (frequency of symptoms, distress, intensity, etc.) [19]. Two findings support this idea. First, there were no significant differences between the clinical and community samples in terms of mean scores on the C-FOCI symptom checklist. This may be because our samples were slightly different to those used in the study by Storch et al. [19]: (1) our sample had different mean ages and a different age range; (2) our clinical sample presented fewer comorbid disorders, and so their degree of severity was lower than in Storch’s sample; and (3) our community sample was larger and more heterogeneous. The second finding derives from the item analysis, which showed a similar percentage of positive OCD symptoms among the clinical and community samples. Results regarding the prevalence of symptoms in the community sample indicated that more than two-thirds of the sample feared losing something valuable, the same rate as that reported by Storch et al. [19]. Additionally, there were three symptoms that were endorsed by nearly 50 % of the community sample: (1) overconcern with order, (2) fear of harm coming to a loved one because the respondent was not careful enough, and (3) reassurance seeking and confessing. Storch et al. [19] also reported rates over 40 % for concern with order and reassurance seeking and confessing. There is a considerable body of data indicating that people without OCD experience “normal” obsessions that are comparable in content to those with OCD [19]. Consistent with our findings, what distinguishes a person with “normal” obsessions from a person with OCD is the degree to which symptoms are frequent, distressing, intense, and provoke efforts to resist. Indeed, some authors have reported certain obsessions to be prevalent in the community, asserting that the number of obsessions and compulsions are endorsed by similar—and in some cases higher—percentages of community subjects relative to pediatric OCD patients (e.g., [42]).

Overall, the psychometric results were encouraging. First, the validity analysis confirmed the factor structure of two independent factors previously suggested by Storch et al. [19]. Confirmatory factor analyses showed that the data fit adequately to the two-dimensional measurement model. Item factor loadings (lambda) for the Severity Scale were, in general, higher than those for the symptom checklist. Regarding invariance, the C-FOCI subscales presented the expected metric invariance between the clinical and community samples, with a RMSEA index below .08 and CFI and NNFI values above .90, indicating an acceptable fit [36]. These data indicate that respondents across clinical and community groups attribute the same meaning to the latent constructs under study (metric invariance).

With respect to reliability, internal consistency for the C-FOCI scores was high (.79–.89), except for the symptom checklist in the clinical sample, where it was acceptable (.73). These data are consistent with the findings of Storch et al. [19] and support the existence of a single consistent dimension for each subscale. C-FOCI scores also showed adequate temporal stability over 4 weeks, suggesting that the instrument could be useful for tracking symptom presence and stability.

Regarding the known-group validity, most symptoms were significantly more frequent in the clinical sample (3, 9, 12, 13, 14, and 16). Specifically, there were seven symptoms that were endorsed by close to or over 50 % of the clinical sample: (1) obsessions involving images of death or horrible things, and (2) fear of losing something valuable, whereas the compulsions concerned (3) ritualized handwashing, cleaning, or grooming, (4) repetitive routine actions, (5) need to touch objects or people, (6) unnecessary rereading or rewriting, and (7) reassurance seeking and confessing (Table 1). Of these, only the symptoms of ritualized handwashing, cleaning, or grooming and reassurance seeking and confessing coincided with those reported by Storch et al. [19] in this respect. Therefore, these data also provide support for the known-group validity of the C-FOCI symptom checklist. Furthermore, mean scores on the C-FOCI Severity scale were higher in our clinical group than in our community sample, with a medium effect size, supporting the scale’s known-group validity [19]. In addition, responses such as “extreme severity” or “severity” across items on the Severity Scale were, on average, more common in the clinical sample than in the community sample, indicating that the OCD sample reported significantly greater severity on each Severity Scale item. This finding is consistent with the reports of Rassin and Muris [43] and Storch et al. [19].

With regard to concurrent validity, the relationships between the C-FOCI scales and three measures with the strongest empirical support, namely the CY-BOCS, OCI-CV and SOCS, support the validity of the C-FOCI. In the clinical sample a stronger positive relationship was found between the C-FOCI Severity Scale and the Total Severity Scale of the CY-BOCS than between the C-FOCI symptom checklist and the CY-BOCS. However, there were also moderate associations between the CY-BOCS and specific anxiety and depression symptom measures. One explanation for this finding is the symptom overlap between depression, anxiety, and OCD disorders, which other studies have also found [5, 42].

Finally, in order to determine the accuracy of the scales for correct classification of children and adolescents with and without OCD, the present study examined the sensitivity and specificity of the C-FOCI scales using ROC analysis. Results indicated a significant area under the curve for the symptom checklist total score, but AUC values of .63 and .77 for the Severity scale. Values between .60 and .70 represent poor discrimination, whereas those between .70 and .80 indicate fair to good discrimination. Thus, the symptom checklist did not offer a good cut-off score for achieving an optimal balance between sensitivity and specificity: the best score would be 7, but this only achieves a sensitivity of 57 % and a specificity of 48 %; however, a score of 6 could, despite its low specificity (38 %), be used to detect children with OCD, due to its greater sensitivity (69 %). In any case, a score of 13 would be more specific for identifying healthy children and adolescents who do not have OCD, offering a prevalence rate for increased symptoms of 4.6 % (n = 49). As for the C-FOCI Severity scale, a score above 7 is recommended because it provides the optimal balance between the percentage of true positives and true negatives in real cases. This cut-off score led to an acceptably moderate percentage of adolescents being incorrectly identified with OCD (specificity = 63 %) and a moderate proportion of adolescents being overlooked in terms of heightened OCD symptomatology (sensitivity = 63 %). However, given that one of the main purposes of the C-FOCI Severity Scale is to serve as a screening measure for OCD, the best cut-off score is 6, with a sensitivity of 73 %, specificity of 53 %, and a Youden’s Index of .27. This C-FOCI score could be useful for screening youth in clinical settings in order to avoid overlooking patients who need further assessment. However, as noted, potential users of this scale could choose a higher or lower cut-off score, depending on their objective. For example, a more conservative cut-off score of 11–12 might be justified when using these subscales for research purposes in order to avoid false positives, with prevalence rates of 7.4 % (n = 79) and 4.6 % (n = 49) respectively (see Table 6). Our sensitivity/specificity results suggest that the C-FOCI is superior to the LOI-CV but similar to other OCD screening instruments [12, 31, 32, 44]. In summary, our prevalence rates with scores of 13 on the symptom checklist and 11–12 on the Severity Scale are consistent with a recent study with a Spanish sample aged from 8 to 12 years which indicated prevalences of 1.8 % for OCD, 5.5 % for subclinical OCD, and 4.7 % for symptomatology (assessed with the Leyton Obsessional Inventory-Child Version Survey) [5]. Moreover, recent studies with Spanish samples using the percentage of participants scoring two standard deviations above the mean score on the OCD subscale on the Spence Children Anxiety Scale (SCAS) found high scores in 4.9 % of the subjects between 8 and 17 years had [45], and in 6.6 % in a sample aged between 10 and 17 [46]. Furthermore, we consider that the possible existence of OCD cases in the community sample, which was as assumed to be free of OCD, might explain the modest sensitivity and specificity in our sample.

Regarding the predictive validity of the C-FOCI, the results from the binary logistic regression analyses indicated that the C-FOCI subscales were significant predictors of OCD diagnosis (classification accuracy of 92 %). This result is consistent with those of the previous study with the C-FOCI [19], which led to the creation of a screening tool that could accurately classify an OCD diagnosis.

Some limitations of the present study should be noted that explain the modest convergence, specificity and sensitivity of the instrument. First, we did not examine the psychometric properties of C-FOCI with a large clinical sample. Furthermore, our clinical sample showed a lower comorbidity rate (52.1 %) than previous epidemiological studies (approximately 80 %) [5]. We stress that our study did not apply an epidemiological design, and that the comorbid disorders found in our OCD sample coincided mainly with those reported by previous studies (highest rates for Generalized Anxiety Disorder, Separation Anxiety Disorder, Depression, Social Phobia, and AHDD and other behavioral disorders) [5]. Second, we included a wide age range encompassing a number of developmental periods including early and late adolescents, but we did not conduct separate analyses for the different age ranges; this must be recognized as a limitation of the study. Third, the instrument’s sensitivity and specificity in differentiating children with and without OCD were examined in a sample of children with the disorder and a non-diagnostically interviewed community sample. Thus, the sensitivity and specificity were modest overall, acknowledging that this restricts the measure’s utility as a screening measure. However, the tool does a good job differentiating between patients with OCD and those without a disorder (AUC = .77), which would indicate the need for further assessment. Fourth, although we describe the process of adapting and translating C-FOCI into Spanish, it should be noted as a major constraint that we did not include the results concerning the translation/cross-cultural validation (although they are available on request). Finally, the external validity of findings is limited, particularly for the US and other English-speaking countries, given the use of Spanish version of the scale using a sample from Spain.

In summary, the C-FOCI is a brief self-report tool (for further details, see the review by Iniesta-Sepúlveda et al. [12]) which offers psychometric properties equivalent to those of other measures. In terms of the Evidence-Based Assessment classification [10], which defines three levels of empirical support, the results of this study and those reported by Storch et al. [19] suggest that the C-FOCI should be considered as approaching a well-established, pediatric OCD-specific assessment instrument, given that at least two published studies by two different research teams would have demonstrated its reliability and validity. Thus, notwithstanding the limitations of this study, the C-FOCI emerges as a promising, valid and useful instrument for the assessment of pediatric obsessive–compulsive symptomatology.

Summary

The C-FOCI is a self-report measure designed to screen the presence and severity of obsessive–compulsive symptoms in children and adolescents, and it presents good psychometric properties. The results of this study involving community and clinical samples confirm the factor structure of two independent factors (symptoms and severity), as well as metric invariance across groups for the symptom checklist and the Severity Scale. The instrument also achieves good reliability in terms of internal consistency and temporal stability, shows modest correlations with other OCD measures, and has modest sensitivity and specificity to detect OCD. Overall, the C-FOCI seems to be a promising screening instrument for assessing the presence and severity of obsessions and compulsions in youth, although further research is needed.