Introduction

The prevalence of Attention Deficit Hyperactivity Disorder (ADHD), a Priority Mental Health Disorder of childhood, is 1.7 % among general population in India [1]. There are a large number of rating scales available for the assessment of ADHD in clinical, school and home settings [2] namely: the Attention-Deficit/Hyperactivity Disorder Test, the Swanson, Nolan, and Pelham Rating Scale–Revised (SNAP-IV), the Werry-Weiss-Peters Activity Rating Scale, the ADD-H Comprehensive Teacher’s Rating Scale (ACTeRS) [3], ADHD Rating Scale-IV, Attention Deficit Disorder Evaluation Scales, and Conners’ Rating Scales-Revised. The ACTeRS has separate threshold scores for boys and girls for diagnosis and intervention planning. Also, in Western cultures ACTeRS has been demonstrated to have adequate psychometric properties in the school and home settings but not the clinical settings [4, 5]. None of the measures for ADHD have been validated for the Indian population, and therefore, the aim of this study is to validate ACTeRS for children with ADHD in clinical settings in India.

Material and Methods

This study was conducted at the Child and Adolescent Psychiatry Unit, Christian Medical College, Vellore, a tertiary hospital that provides services to a large geographical region. Data was collected from clinical records maintained in the facility either as part of the day care or residential therapy program or drop-in outpatient clinic.

ADD-H Comprehensive Teacher’s Rating Scale (ACTeRS) [3] is a 24-item questionnaire for children 5–12 y of age, and measures 4 areas of behaviors i.e., attention deficit, hyperactivity, oppositional behavior and social skills. Higher scores in attention and social skill domains suggest better attention span and social skills, where as higher scores in hyperactivity and oppositional behaviors indicate poor motoric control and higher levels of behavioral problems, respectively. This scale was the measure for validation. The International Classification of Diseases, Edition-10 (ICD-10) for Mental and Behavioural Disorders based clinical interview with emphasis on Hyperkinetic disorders [Disturbance of activity and attention (F90.0), Hyperkinetic conduct disorder (F90.1), Other hyperkinetic conduct disorder (F90.8) and Hyperkinetic disorder, unspecified (F90.9)] and Attention Deficit disorder without hyperactivity (F98.8) was the reference standard.

Swanson, Nolan, and Pelham–IV (SNAP-IV) [6] is a 90-item scale, for children of 6–12 y of age, which includes 18 symptoms for the Diagnostic and Statistical Manual, Edition-IV (DSM-IV) diagnosis of ADHD and 8 symptoms contained in the DSM-IV diagnosis of oppositional defiant disorder (ODD). These items are rated on a 4-point scale in 3 domains: inattention, hyperactivity/impulsivity, and oppositional. This was the measure for assessing the convergent validity of the attention deficit, hyperactivity and oppositional subscales of ACTeRS.

Vineland Social Maturity Scale (VSMS) [7] gives the adaptive behavior in eight areas: self help general, self help dressing, self help eating, socialization, self direction, communication, locomotion and occupation and was used to evaluate the convergent validity of the social skill subscale of ACTeRS.

The Childhood Autism Rating Scale (CARS) [8] is a 15 item behavior-rating scale designed to detect and quantify symptoms of autism and was used to evaluate the divergent validity of the index measure.

The illness details and ICD-10 diagnosis contained within the medical records were made by the multidisciplinary treating team much ahead of the time when this data were collected. Thus, the data extracted were from the psychiatry case-notes made by the psychiatrists, psychological assessment notes, occupational therapy details or speech therapist’s notes. The data were extracted by a medical student as well as a postgraduate trainee in psychiatry, and a consultant psychiatrist resolved the data disagreements. The data was protected by reversible anonymisation and restricted availability of the data to others. The Institutional Review Board of Christian Medical College reviewed and gave permission to conduct the study.

Sensitivity and specificity for various ACTeRS cut-off scores (only the attention and hyperactivity subscale scores) were calculated in order to determine the optimal screening as well as diagnostic threshold with Receiver operating characteristic (ROC) analyses and contingency tables. The test-retest reliability of ACTeRS was examined with the intra class correlation. For internal consistency, Cronbach’s α coefficient was calculated. To determine the convergent validity the subscales of ACTeRS was correlated with the subscales of SNAP-IV or VSMS. The criterion validity was assessed with the concordance rate between ACTeRS threshold score and the ICD-10 based diagnosis using the Cohen’s Kappa test. Divergent validity was calculated by correlating total ACTeRS score with total CARS score, as it was hypothesized that SNAP-IV diagnosis of ADHD would be closely related to the ACTeRS scores than CARS scores measuring autism. The Factor structure of ACTeRS was demonstrated by principal components analysis with varimax rotation. Data was analysed using SPSS software version 19.

Results

Among the 110 participants, 39 had average intelligence with an Intelligence quotient (IQ) or Developmental quotient (DQ) above 71. The remaining 71 children had compromised intelligence and their IQ or DQ ranged from 39 to 70. The mean (SD) age of the participants was 6.89(2.8) with a range of 2 to 16 y. Majority of the participants were boys (boys vs. girls = 77.3 % vs. 22.7 %). The mean (SD) ACTeRS was 61.5(14.2) with a range of 22 to 86 and SNAPS-IV as well as CARS score were 32.90 (12.41) with a range of 14 to 55 and 26.14(11) with a range of 9 and 45 respectively.

For the sensitivity and specificity, differing cut-off points for the ACTeRS were tested. Table 1 summarizes these results. A score of ≥61 in ACTeRS achieved sensitivity of 85.51 % and specificity of 90.24 % making it appropriate for a diagnostic use to establish ICD-10 diagnosis of ADHD. The area under curve (AUC) in the ROC for the ACTeRS was 0.94 with a standard error of 0.1.

Table 1 Diagnostic accuracy of ACTeRS against the gold standard of ICD-10 based clinical diagnosis
Full size table

The test-retest reliability at a mean (SD) period of 5.4(7.1) wk for ACTeRS was 0.87. The internal consistency for the ACTeRS (whole scale) was high (α = 0.80) and for the subscales also the internal consistency was high at 0.91 (Attention), 0.93 (Hyperactivity), 0.89 (Social Skills), and 0.92 (Oppositional). The domain-total correlation between the total ACTeRS score and ACTeRS items ranged from 0.03 to 0.72. None of the 24 items was assigned a score of 0 by more than half of the children with ADHD in this study suggesting that the Content validity was appropriate to their Attention Deficit Hyperactivity Disorder. The convergent validity between the ACTeRS and SNAP-IV attention deficit subscale, hyperactivity subscales and oppositional subscales showed a moderate correlation (r = 0.60, P = 0.005; r = 0.49, P = 0.02; r = 0.58, P = 0.008, respectively). However, the convergence between the ACTeRS subscale of social skills and VSMS social age showed a low correlation (r = 0.30, P = 0.01). As part of the criterion validity analysis, there was a high concordance rate of 82.52 % (ADHD based on ICD-10 vs. ADHD based on ACTeRS score of ≥ 61 = 69/110 vs. 72/110) [Cohen’s κ = 0.78, P = 0.001] between the ACTeRS and reference standard of ICD-10 diagnosis in identifying ADHD among the children. Divergent validity calculated by correlating ACTeRS total scores to the CARS total scores showed non-significant associations (r = −0.36; P = 0.1) demonstrating that the ACTeRS discriminates ADHD from other childhood psychiatric disorders like autism.

To investigate the construct validity, authors explored the factor structure of the items in the ACTeRS. They extracted those factors with an eigen value of 1 and thus a 4-factor structure was derived. There was no ACTeRS item that did not achieve the required factor loading to load on to atleast one factor, but two of the items, “Persists with task for reasonable amount of time” and “Functions well in classroom” cross-loaded on to the Attention deficit/ Hyperactivity factors and Adaptive behavior/Hyperactivity factors respectively. The factor structure details are summarized in Table 2. This 4-factor structure explained 65 % of the variance.

Table 2 The 4-factor structure of ACTeRSa
Full size table

Discussion

This study demonstrates that ACTeRS has good psychometric properties among children with intellectual disability (ID) and supports the study done earlier [5]. The earlier study had focussed primarily on the reliability of ACTeRS in children with compromised intelligence. The authors also documented the diagnostic accuracy and validity of ACTeRS in an Indian setting.

The internal consistency of the various versions of ACTeRS has varied from 0.80 to 0.93 in the index study and is similar to the very high at 0.92 to 0.97 reported before [2]. For the whole ACTeRS the internal consistency was only 0.80, but for the subscales of attention and hyper activity it was high at 0.91 and 0.93 respectively. As the whole scale had the other constructs of oppositional behavior and social skills were also incorporated the overall internal consistency could have been low. The test-retest reliability in the index study for the teacher version was 0.87, which is slightly higher than the intra-class correlation coefficient (ICC) of 0.78 to 0.82 for the parent version [9]. About the construct validity, a 4-factor structure has been reported which is akin to the present factor structure as well. Parent and self-reported versions have been documented to have 3 and 5-factor structures respectively [9]. It was found that the ACTeRS was able to discriminate autism from ADHD, similar discriminatory quality has been noted against non-clinical children and children with learning disorders [9]. There is no test-retest or diagnostic accuracy and criterion validity data on the teacher version or on any version of ACTeRS respectively in the literature to compare with. The limitations are that the validation study was done in a tertiary hospital and thus compromises it’s use in other settings. However, unlike many other checklists for ADHD, ACTeRS is a brief measure and can be of help in countries like India where there is low human resources for mental health.