Introduction

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder manifested by a persistent inability to social communication and social interaction and repetitive behavioral patterns, interests, or activities (American Psychiatric Association, 2013). In DSM-IV, Autistic Disorder and Asperger Syndrome are defined under the title of pervasive developmental disorders (PDD). Asperger syndrome (AS) differs from Autistic Disorder in the sense that there is no delay in development of the language and cognitive domains; however, there are significant problems in social interaction (APA, 2000). In the Fifth Edition of the diagnostic and statistical manual of mental disorders (DSM-5), all ASDs were collected in the ASD diagnostic category (APA, 2013). The definition of high functioning autism (HFA), which is not included in the ICD and DSM systems, is used to define the clinical picture of autism without intellectual disability (Montgomery et al., 2016). In the DSM-5 diagnosis of ASD, the specifiers "without accompanying intellectual impairment" and "without accompanying language impairment" can function to indicate AS/HFA cases (APA, 2013). Social (pragmatic) communication disorder (SCD) is a new diagnostic category included under Communication Disorders in the Neurodevelopmental Disorders section of the diagnostic and statistical manual of mental disorders, fifth edition (DSM-5) (APA, 2013). With this change in DSM-5, SCD has been included in the classification. With this, it was aimed to prevent the losses that individuals who were in the DSM-IV-TR diagnosis group would cause by not being diagnosed in the new diagnosis system (Swineford et al., 2014). The diagnoses of SCD and AS are similar in terms of the requirement for deficiency in social skills. However, SCD individuals do not have limited interests, stereotyped-repetitive behaviors, and sensory abnormalities (APA, 2013).

Baron-Cohen et al. (2001), developed the Autism Spectrum Questionnaire (AQ). The adult form is developed to measure the degree of autistic traits based on self-report. The adolescent form of the scale is also a useful tool in detecting adolescents in the transition zone between ASD and typical development (Baron-Cohen et al., 2006). Adolescents with autistic characteristics may have a suspicious awareness of themselves due to their social deficiencies, and therefore the AQ—Adolescent form is designed as a parent-report scale (Baron-Cohen et al., 2006). Poon et al. (2020) compared the psychometric properties of the self and parent‑report versions of AQ‑Adult with similar concerns. In this study parent-report version showed significantly stronger psychometric properties than the self-report version.

This study aims to determine the validity and reliability of the Turkish version of the Autism Spectrum Questionnaire—Adolescent form. This tool measures the degree of subthreshold autistic features in individuals with average intelligence in the community and clinical group and helps to identify AS/HFA adolescents.

Methods

Participants and Procedure

Approval for the study was obtained from the Non-Interventional Clinical Research Ethics Committee of Dokuz Eylül University and was dated 19.07.2018, numbered 2018/18–30. Patients in the 11–18 age group who were followed up in Dokuz Eylul University Faculty of Medicine Child and Adolescent Psychiatry Policlinic between 1 July 2018 and 15 December 2018 were included in the study group. It consists of 400 adolescents and their parents, including 80 diagnosed with ASD (without accompanying language impairment and mental impairment), 71 diagnosed with other psychiatric disorders (Obsessive Compulsive Disorders, Major Depressive Disorders, Social Phobias), and 249 healthy controls.

Different child psychiatrists in our clinic referred 92 patients who were diagnosed with Asperger Syndrome according to DSM-IV-TR or who were diagnosed with ASD—without language/mental impairment” according to DSM-5 during outpatient follow-up. Three patients diagnosed with Asperger's Syndrome a long time ago were not included in the study groups because they did not meet the diagnostic criteria during the study's examinations. Seventy-one of the referred patients had an intelligence test performed during follow-up. We applied WISC-R to 19, and WAIS to 2 patients of 21 patients who did not have an intelligence test during the follow-up period. Nine were excluded from the study after scoring under 70 in the test due to mental retardation. Thus, the remaining 80 patients were included in the study in the AS/HFA group.

For the psychiatric disorder (PD) group, 87 patients who were diagnosed with major depressive disorder (MDD)/obsessive–compulsive disorder (OCD)/social phobia (SP) were referred to the study by other child psychiatrists during outpatient clinic follow-up. These patients were evaluated as having average intelligence on clinical examination. To confirm the diagnosis, we applied the Kiddie Schedule for the Present and Lifetime Version of Affective Disorders and Schizophrenia (KSADS-PL) to these patients. 9 out of 87 patients were excluded from the study because they did not meet the diagnostic criteria during the study or had subthreshold symptoms. Seven patients were excluded from the study because they did not attend the necessary repeat interviews. Thus, the remaining 71 patients were included in the PD diagnosis group (35 MDD, 18 OCD, and 18 SF).

The healthy control (HC) group consisted of 274 adolescents who admitted to our clinic for consultancy reasons but did not meet any psychiatric disorder's diagnostic criteria and adolescents who reached us via the hospital's information board. Twenty-five of these adolescents were not included in the study due to missing data/not answering all of the questions. Thus, the remaining 249 adolescents were included.

The mean age of adolescents in the study group is 13.80 ± 1.76 in all cases. The AS/HFA group's mean age was 13.85 ± 1.97 and consisted of 16 (20%) female and 64 (80%) male patients. The mean age of the PD group was 14.06 ± 1.98. It consisted of 43 (60.6%) female patients and 28 (39.4%) male patients. The healthy control group's age was 13.71 ± 1.62 and consisted of 129 (51.8%) female and 120 (48.2%) male patients. There was no significant difference in age between the three groups (One-way Anova Test), but a significant difference was found in terms of gender (Pearson Chi-square Test p < 0.001). 80% of the AS/HFA group was male, 39.4% of the PD group was male, and 48.2% of the controls were male. In our study group (AS/HFA), the male/female ratio was higher, in line with the literature that ASDs are more common in males.

Measures

The AQ—Adolescent’s Version

The AQ-Adolescent is a scale that measures autistic characteristics in adolescents based on the parent report. It evaluates in five different areas consisting of 10 items, and these areas are; “Social skills” (1, 11, 13, 15, 22, 36, 44, 45, 47, 48); “Attention switching” (2, 4, 10, 16, 25, 32, 34, 37, 43, 46); “Attention to detail” (5, 6, 9, 12, 19, 23, 28, 29, 30, 49); "Communication" (7, 17, 18, 26, 27, 31, 33, 35, 38, 39) and "İmagination" (3, 8, 14, 20, 21, 24, 40, 41, 42, 50). AQ is a four-point Likert scale with the responses "Definitely Agree," "Slightly Agree," "Slightly Disagree," and "Definitely Disagree." If the respondent parent records autistic-like behaviors as mild/strong, that item gets 1 point. For an individual who is expected to score high on the scale, such as AS or HFA, approximately half the items were composed of a "disagree" response and half an "agree" response. ‘Definitely disagree’ or ‘slightly disagree’ responses scored 1 point for items 1, 3, 8, 10, 11, 14, 15, 17, 24, 25, 27, 28, 29, 30, 31, 32, 34, 36, 37, 38, 40, 44, 47, 48, 49, 50 and ‘definitely agree’ or ‘slightly agree’ responses scored 1 point for items 2, 4, 5, 6, 7, 9, 12, 13, 16, 18, 19, 20, 21, 22, 23, 26, 33, 35, 39, 41, 42, 43, 45, 46 (Baron-Cohen et al., 2006). The range of possible scores is 0–50. The scoring key form prepared to facilitate the calculation of the scale was used. This form shows which two out of four possible responses will get 1 point and the other two will get 0 points.

AQ-Adolescent Translation Process

The AQ's items and structure of the Adolescent form are the same as in the adult form, but it is designed to be scored by the parent. There are many translations and psychometric studies of the adolescent, adult, and child forms of the scale, such as French (Sonié et al., 2013), Japanese (Wakabayashi et al., 2006, 2007), Dutch (Hoekstra et al., 2008), Australian (Broadbent et al., 2013) and Turkish (adult) (Kose et al. 2010) versions. Before our study, we obtained permission via e-mail from the study group that developed the scale. The English version of the Adolescent Autism Spectrum Quotient (Ages 12–15 Years) and it’s Scoring Key was downloaded from the Autism Research Center web page (www.autismresearchcentre.com/tests/). Later, a researcher working as the child and adolescent psychiatry resident did the first translation. The translation was reviewed and revised by three child psychiatrists who spoke good English and Turkish. A native speaker of English, who also spoke Turkish, back translated the scale from Turkish to English. All items were examined one by one and converted into a form to be easily understood and culturally appropriate.

The Autism Spectrum Screening Questionnaire

We used the Autism Spectrum Screening Questionnaire (ASSQ), a parent rating scale for concurrent validity in our study. It is a 27-item scale developed by Ehlers et al. (1999), which aims to measure autistic symptoms in 6 to 17 years old children. The questionnaire is scored with the following alternatives for each item: “No” (0 points), “Somewhat” (1 point) and “Yes” (2 points). The minimum score is 0 and the maximum 54. Köse et al. (2017) determined the psychometric properties of the Turkish version of the ASSQ.

The Kiddie and Young Adult Schedule for Affective Disorders and Schizophrenia Present and Lifetime Version

The Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL) is a semi-structured diagnostic interview in children and adolescents aged 6 to 18 years according to DSM-IV criteria (Kaufman et al., 1997). The K-SADS Turkish validity and reliability study was conducted by Gökler et al. (2004).

Statistical Analysis

Frequencies (N) and percentages (%) are used to present the descriptive characteristics of the data; numerical variables are represented as the mean ± standard deviations or Median (Minimum–Maximum). Whether the data were normally distributed was analyzed through visual techniques (histograms), descriptive techniques (coefficient of variation, skewness, and kurtosis), and analytical methods (Kolmogorov–Smirnov test). Internal consistency was evaluated by the Cronbach α values of AQ-Adolescent and subscales. For test–retest reliability, we calculated the Spearman correlations by re-applying the scale to the parents of 40 adolescents randomly selected from the AS/HFA group 14 days later. Structure Validity, Criterion Validity, and Discriminant Validity were used to show the AQ-Adolescent's validity. KMO and Barlett's test of sphericity was used for data suitability for explanatory factor analysis. Exploratory factor analysis (EFA) was used to show structure validity. After the exploratory factor analysis, we performed a confirmatory factor analysis (CFA). In order to evaluate the criterion validity of the AQ, the correlations between AQ-Adolescent and its subscale scores and ASSQ scores were calculated with the Spearman correlation test. Kruskal Wallis Test was used to show the difference of AQ scores between patient and healthy groups. In this way, discriminant validity was shown. A ROC Curve Analysis was applied to determine the cut-off points of the scale. IBM SPSS 21 for Windows used for data analysis with statistical significance set at p < 0.05.

Results

Reliability Findings

Item Analysis

In Table 1 the results of calculated scoring rate for each item of AQ-Adolescent in the AS/HFA, PD, HC groups and scoring rate by gender are shown. The rate of answers in the autism-like direction can be seen in this table.

Table 1 Rate of the AS/HFA, PD and HC groups scoring on each item
Full size table

Internal Consistency

We found the internal consistency Cronbach α value of 0.829 for AQ-Adolescent. This value represents that AQ-Adolescent has achieved a “good” reliability level. Cronbach α values were determined for Social skill (0.570), Attention switching (0.427), Attention to detail (0.577), Communication (0.608) and Imagination (0.686) subscales. Ordinal coefficient alpha is a reliability index estimates the internal consistency for scales involving ordinal data, using the polychoric correlation matrix for its estimation (Zumbo et al., 2007). Ordinal alpha is an alternative to Cronbach's Alpha that is more accurate with Likert-scale responses. Since AQ-Adolescent is a likert type scale, we also calculated the ordinal alpha in addition to the cronbach alpha value. We found ordinal alpha value of 0.90 for AQ-Adolescent. Subscale ordinal α values were determined for Social skill (0.72), Attention switching (0.59), Attention to detail (0.70), Communication (0.74) and Imagination (0.83) subscales.

Test–Retest Reliability

AQ-Adolescent was re-administered to 40 adolescents with AS/HFA 14 days later and the test–retest values (Spearman correlation analysis) were determined. “Very Strong” significant correlation values were detected in AQ-Adolescent total scores (r = 0.953; p < 0.01) and all sub scales including the Social skill (r = 0.976; p < 0.01), Attention switching (r = 0.928; p < 0.01), Attention to detail (r = 0.959; p < 0.01), Communication (r = 0.922; p < 0.01), and Imagination (r = 0.885; p < 0.01) (Table 2).

Table 2 Test–retest correlations of AQ-Adolescent and its subscales (N = 40)
Full size table

Validity Findings

Structure Validity/Factor Analysis

We conducted explanatory factor analysis (EFA) to assess structure validity. According to the factor analysis results, The Kaiser–Meyer–Olkin value of the scale was determined to be 0.80. In factor analysis, we obtained a four-component structure that explains 30.31% of the total variances and includes 41 items. These results support the construct validity of the scale (Table 3). Items 37 (D), 43 (D), 46 (D), 2 (A), 4 (A), 14 (C), 24 (I), 36 (S), 48 (S) were not loaded over 0.300 coefficient in any of the 4 factors. Since these nine items were not loaded on any of the factors, the four-factor version of the scale was considered the AQ-Adolescent 41-item version. The Cronbach α values of this 41-item version and Factors 1, 2, 3 and 4 were 0.816, 0.807, 0.698, 0.629, and 0.453, respectively.

Table 3 Factor loads of 41 items of AQ-adolescent applied to all cases in exploratory factor analysis (EFA)
Full size table

In Confirmatory Factor Analysis (CFA), CMIN/DF: found compatible with Model data as 1.817 (CFI: 0.739 GFI: 0.850). Item 27 (p: 0.004) for unstandardized coefficients, Item 28 (p: 0.016) for other items p < 0.001. Due to the low variance, the standardized coefficients AMOS did not run the program. Standardized coefficients could not be evaluated in CFA due to variance. CMIN/DF: 1.817 was found to be compatible with the Model data. CFI: 0.739 GFI: 0.850 and RMSA 0.045 models were found to be compatible. Item 27 (p: 0.004) for unstandardized coefficients, Item 28 (p: 0.016) for other items p < 0.001. Standardized coefficients could not be evaluated due to low variance. Model fit index can be seen in Fig. 1. There was minimal support for CFA models.

Fig. 1
figure 1

Model fit index in CFA

Full size image

Criterion Validity

We found a ‘strong’ positive significant correlation between AQ-Adolescent total scores and ASSQ total scores. ‘Moderately’ significant positive correlation between the ASSQ score and Social skill, Attention switching, Attention to detail, Communication and Imagination subscale scores and a ‘strong’ positive significant correlation between the ASSQ score (Table 4).

Table 4 Results of correlation analysis between AQ-Adolescent and subscale and ASSQ scores
Full size table

Discriminant Validity

We evaluated whether the tool discriminates those diagnosed with AS/HFA or PD from healthy controls. We found a significant difference between all groups for AQ-Adolescent total score (Kruskal Wallis Test p < 0.001). Scores according to the groups were ranked as AS/HFA group > PB group > HC. We found no significant difference only in the ‘Attention to detail’ subscale. The distribution of the AQ-Adolescent scale and subscale scores by groups is shown in Fig. 2.

Fig. 2
figure 2

The distribution of the AQ-Adolescent scale and subscale scores by groups

Full size image

ROC Analysis and Cut-Off Value

The ROC curve was drawn to assess the predictive power of AQ-Adolescent, ASSQ and AQ-Adolescent (41 items version). We showed the areas under the curve in Fig. 3. We found the area under the ROC analysis curve as 0.998 (95% CI 0.994–1) for AQ-Adolescent, 0.996 (95% CI 0.990–1) for AQ-Adolescent (41 items) and 0.983 (95% CI 0.969–0.997) for ASSQ (Table 5). Sensitivity and specificity values for possible cut-off points are shown in Table 6. 24 for AQ-Adolescent and 21 for OSA-Adolescent (41 items) were the most appropriate cut-off points. Since it is difficult to diagnose and treat the AS/HFA patient group, the selection of sensitivity and specificity values for the cut-off point was made considering the high specificity value.

Fig. 3
figure 3

Power of AQ-Adolescent and ASSQ scores in predicting the diagnosis of AS/HFA with ROC analysis

Full size image
Table 5 Areas under the ROC curve for AQ-Adolescent and ASSQ Scales
Full size table
Table 6 Sensitivity and specificity values according to possible cut-off scores
Full size table

Discussion

This study's main purposes were to examine if the AQ-Adolescent could be applied to Turkish adolescents and compare our results with other adaptation studies' data. According to our results, the scale is valid and reliable for the Turkish language and culture. We concluded that the Turkish version of the AQ-Adolescent could be used safely as a screening tool in this age group.

In our study in the item analysis, the only item scored more frequently by the PD and HC groups compared to the AS/HFA group was 43. But this difference was statistically insignificant. In the original UK study of the AQ-Adolescent, the items numbered 29 and 30 were scored at a higher rate by the HC group compared to the patient group (Baron-Cohen et al., 2006). The French study of the AQ-Adolescent showed that items numbered 29, 30, and 49 were scored higher by healthy control subjects compared to the AS/HFA and PD groups. In the same study, they also found that controls scored items 6 and 14 at a higher rate than the PD group (Sonié et al., 2013). In the validity and reliability studies of AQ forms for all age groups (adult, adolescent, child), they found many times which item 29 was scored a higher rate by healthy controls (Baron-Cohen et al., 2006; Sonié et al., 2013; Wakabayashi et al., 2006; Wakabayashi et al., 2007). There is an argument in the literature about removing item 29 from the revised AQ, since it is not valid in discrimination (Wakabayashi et al., 2006).

In our study the Cronbach α value was 0.829 for all items of AQ-Adolescent 0.570 for Social skill, 0.427 for Attention switching, 0.577 for Attention to detail, 0.608 for Communication, and 0.686 for Imagination. Baron-Cohen et al. (2006) reported Cronbach α values of 0.79 for AQ-Adolescent, 0.88 for social skill, 0.76 for attention switching, 0.66 for attention to detail, 0.82 for communication, 0.81 for imagination. In the French Study these values were 0.88 for AQ-Adolescent, 0.91 for social skill, 0.91 for attention switching, 0.95 for attention to detail, 0.90 for communication, 0.92 for imagination (Sonié et al., 2013). Our results showed good internal consistency of the AQ-Adolescent and AQ-Adolescent (41 items), similar to the other studies. But five subscales of the AQ-Adolescent and four factors of the AQ-Adolescent (41 items) had ‘poor’ or ‘questionable’ Cronbach alpha values except Factor 1 (0.807). Internal consistency was also evaluated by ordinal alpha, and it was 0.90. When we examined the studies conducted for all versions of the scale in the literature, we could not find any other studies that evaluated ordinal alpha.

In the test–retest, the AQ-Adolescent total score's correlation value was 0.953 (p < 0.01) in our study. Baron-Cohen et al. (2006) re-applied the scale to the parents of 15 patients and found the correlation value as r = 0.92 (p < 0.001). In another study AQ-Adult was re-applied to 80 of 300 control parents of typically developing children after a period of 6 months, and the correlation was 0.79 (Ruta et al., 2012). Correlation values were found as 0.78 in the Dutch study (Hoekstra et al., 2008), 0.95 in the Australian study (Broadbent et al., 2013), 0.72 in the Turkish study (Kose et al. 2010), and 0.87 in the Japanese study (Wakabayashi et al., 2006) for the adult forms. Our study determined AQ-Adolescent test–retest reliability and showed that AQ-Adolescent had as “very strong” significant correlation values as in other studies in the literature.

For the adolescent form, there is no factor analysis either in the original scale study or in other adaptation studies as far as we know. Our study is the first study that performed a factor analysis of the adolescent form. In the factor analysis for construct validity, we obtained the four-factor structure including 41 items that explains 30.31% of the variance. Sixteen items were loaded on the first factor explaining 11.27% of the variance and ten items were loaded on the second factor explaining 7.11% of the variance. Eight items were loaded on the third factor explaining 6.83% of the variance and seven items were loaded on the third factor explaining 5.10% of the variance. Analysis in terms of language and culture revealed that clustered items were in domains including 'communication and imagination' for Factor 1, 'social skills' for Factor 2, 'special attention-interest' for Factor 3, and 'detailed awareness' for Factor 4.

Studies, including factor analysis, are available for adult and child forms of the scale. Austin (2005) defined a three-factor structure of the AQ-Adult that explains 28% of the variance in their study. They suggested the category names of (Poor) Social skills, Details/Patterns, and (Poor) Communication/Mind reading. Hurst et al. (2007) examined the reliability and factor structure of the AQ and found results supporting the three-factor model of the AQ (26-items) proposed by Austin. Auyeung et al. (2008)'s factor analysis provided support for four of the five AQ-Child design subscales. The four-component solution explained a total of 40% of the variance. Gomez et al. (2019) examined through the implementation of confirmatory factor analyses the support for both the five- and four-factor models previously suggested for AQ-Child. Their findings indicated most support for a four-factor model, with factors reflecting “mind reading”, “social skills”, “attention to details”, and “imagination”. Hoekstra et al. (2008) evaluated the factor structure of the Dutch version of the AQ-Adult and they identified a two-factor model, consisting a factor including the ‘Attention to detail’ subscale and a higher order ‘Social interaction’ factor including other AQ-subscales. In the study of Kose et al. (2010), a structure with three factors was found explaining the first 8.6%, the second 6.5%, the third 4.4% of the variance; and they termed the three domains of the 38-item version of the AQ-Adult as follows: Communication-mind reading, Details and Social skills. Hurst et al. (2007) stated that the psychometric problems of the AQ are not only the result of cultural differences in item interpretation.

In our study, the construct validity of the scale was supported by factor analysis. However, as in other studies, the construct's validity containing five subscales in the original form of the scale could not be demonstrated. Besides, we found some invalid values in our evaluation of the scale's factor structure, especially in the confirmatory factor analysis. In the literature review, we came across many studies with similarly poor results regarding the scale. In many studies in which psychometric assessments of AQ were made, the number of factors defined in the scale and the psychometric properties of these factors led to significant conflicts. According to all these results, there were clearly some problems in the factor structure of AQ. Studies found insufficient internal consistency generally associated with the original factors, and these findings suggest that factors were poorly defined during the construction phase of the scale. English et al. (2020) focused on this problem in the factor analysis of the scale. They conducted a series of repetitive factor analyses in their studies on a larger population and agreed on the problem's existence.

We conducted the correlation between the scores of the Turkish version of ASSQ, a tool developed to screen for symptoms similar to AQ, and the scores of AQ-Adolescent total and subscales with the Spearman correlation test. There is a 'strong' significant correlation between AQ-Adolescent and ASSQ (r = 0.74). There are few studies in the literature assessing criterion validity using a similar scale. In the study in which the AQ-Adult form was adapted to Korean, they found a negative coefficient correlation with the Empathy Quotient (− 0.66). They interpreted as a result showing that the scale can measure subthreshold autistic symptoms (Ko et al., 2018).

We determined whether the scale discriminated those with AS/HFA or PD from the HC group. We found a significant difference between all groups (p < 0.001). There was a significant difference between all scores in AS/HFA > PD > HC, but no significant difference was found between PD and HC only in the ‘Attention to detail’ subscale. This data showed that the scale could distinguish AS/HFA and PD groups from healthy controls. There are other studies in the literature that have discriminant validity (Baron-Cohen et al., 2006) (Wakabayashi et al., 2007). Similar to our results, in these two studies, they founded that the scale has an intergroup distinguishing feature in the AS/HFA group > PD group > healthy control groups (Hoekstra et al., 2008; Sonié et al., 2013).

A 24 and above cut-off score was accepted for the AQ-Adolescent with a sensitivity of 0.975 and a specificity of 0.992. In the cut-off score assessment of AQ-Adolescent initial validation study, an AQ score of 30 + was considered as a useful cut-off score in future screening studies (Baron-Cohen et al., 2006). In Sonié et al.'s study (2013), the area under the curve was calculated as 0.98 (95% CI 0.96–0.99) in the ROC analysis. They reported that the cut-off score of 26 could distinguish the autism group from healthy controls with 0.89 sensitivity and 0.98 specificity. The cut-off points calculated in our study are close to the cut-off score reported in this study, which is the only adolescent study found in the literature except for the initial validation study of the instrument.

For a patient group such as AS/HFA, early diagnosis is critical for early social and educational intervention. This cross-sectional case—control study confirmed that the AQ device could be used as a Turkish culture screening tool. Besides, although the scale was developed as a screening tool rather than a diagnostic tool, it appears to be very useful in distinguishing individuals with AS/HFA from ones with PDA or healthy individuals. Strong values cannot be shown in EFA and CFA due to the scale's original structure. However, we present a high validity and reliability results with test–retest and criterion validities in this study. One of the strengths of our study is that we tested the construct validity with both explanatory and confirmatory factor analysis, although it was not done during the development of the scale.