Attention-Deficit/Hyperactivity Disorder (ADHD) is the most prevalent psychiatric diagnosis in children, affecting approximately 6.4 million children in the United States (CDC 2011). ADHD is characterized by sustained and pervasive patterns of abnormalities in attention, deficits in inhibition, and excessive hyperactive behavior (APA 2000; APA 2013). The manifestation of this diagnosis shares considerable overlap with other disruptive behavior disorders, such as Oppositional Defiant Disorder (ODD) (Kuhne et al. 1997; Stefanatos and Baron 2007; Pliska 2011). In fact, ODD and ADHD are highly comorbid, with comorbidity estimates nearing 60 % (Semrud-Clikeman and Ellison 2009). Differential diagnosis of ADHD and ODD remains problematic in clinical practice (Biederman et al. 1991; Stefanatos and Baron 2007; Connor and Doerfler 2009; Connor et al. 2010). In addition to high rates of comorbidity between ADHD and ODD, individuals with ADHD are also more predisposed to difficulties socially interacting with their peers in appropriate ways, age appropriate emotional functioning, and academic functioning (Smith et al. 2009). These complex presentations of symptoms can prompt caregivers to seek mental health treatment or educational accommodations. However, unclear or inaccurate diagnosis can obfuscate appropriate treatment planning for individuals seeking services and complicate shared understanding of the etiology and progression of each respective disorder (Stefanatos and Baron 2007; Connor and Doerfler 2009).

Diagnosis of both ADHD and ODD relies heavily on the observation of symptoms across multiple environments (APA 2000; APA 2013), which are primarily composed of teacher and parent ratings (Stefanatos and Baron 2007; Smith et al. 2009). In conjunction with other assessment measures and data collection techniques, rating scales provide an objective method of comparing potentially abnormal childhood behavior to normative data and comparing raters across settings. However, agreement between parent and teacher ratings of ADHD symptoms can vary, complicating the clinician’s ability to formulate a definitive diagnosis and treatment plan (Antrop et al. 2002). Low to moderate agreement across parent and teacher rating scales has been well documented in the literature (Antrop et al. 2002; Murray et al. 2007; Sollie et al. 2013; Narad et al. 2015); however, less is known about how parent and teacher ratings may differ. One possibility is that parents and teachers may be differentially susceptible to negative halo effects.

Halo effects occur when a child who is displaying one discrete behavior (e.g., hyperactivity) is rated as exhibiting other behaviors (e.g., oppositionality) despite those behaviors not being evident (Schachar et al. 1986; Abikoff et al. 1993; Stevens et al. 1998; Jackson and King 2004; Hartung et al. 2006; Hartung et al. 2010). The artificial rating of one symptom in the presence of another may be unidirectional or bidirectional in nature, resulting in the excessive inflation of one or multiple symptoms (Jackson and King 2004; Hartung et al. 2006; Hartung et al. 2010). Specifically, a halo effect is unidirectional if its presence leads to another symptom being falsely endorsed (e.g., the presence of hyperactivity inflates ratings of oppositionality but the presence of oppositionality does not inflate ratings of hyperactivity). A halo effect is considered bidirectional if the presence of a primary symptom inflates ratings of a secondary symptom and the presence of the secondary symptom also inflates ratings of the primary symptom (e.g., the presence of inattention inflates ratings of oppositionality and the presence of oppositionality inflates ratings of inattention).

Halo effects have been demonstrated in teacher and college student ratings of ADHD and ODD symptoms (Abikoff et al. 1993; Stevens et al. 1998; Hartung et al. 2006; Jackson and King 2004; Hartung et al. 2010). For example, in Jackson and King’s (2004) study, child actors portrayed both boys and girls with ADHD, ODD, or developmentally appropriate behavior in videotaped vignettes. Teachers rated the children in the videotaped vignettes and their ratings illustrated a bidirectional halo effect. Specifically, the portrayal of oppositional behavior resulted in higher ratings of inattention and hyperactivity. In addition, the portrayal of inattention and hyperactivity resulted in higher ratings of oppositionality.

More recent studies also confirmed the presence of bidirectional halo effects. In Hartung et al. (2006), college student raters were presented with written vignettes that depicted ADHD Combined Type (ADHD-CT), ADHD Predominately Inattentive Type (ADHD-PI), ODD, or Major Depressive Disorder (MDD). Results revealed bidirectional halo effects, with inattention and hyperactivity inflating ratings of oppositionality and oppositionality inflating ratings of hyperactivity and inattention. Hartung et al. (2010) sought to extend the research to determine if specific symptoms were more susceptible to halo effects than others.

Hartung et al. (2010) utilized written vignettes that depicted ADHD-CT, ADHD-PI, ODD, and a typically developing child. Ratings from college students indicated unidirectional and bidirectional halo effects in addition to evidence that some symptoms were especially susceptible to halo effects. Specifically, Hartung et al. (2010) found two inattention symptoms that were more likely to be endorsed in vignettes in which inattention was not targeted (i.e., Doesn’t listen when spoken to directly and Avoids, dislikes or is reluctant to engage in work that requires sustained mental effort). Similarly, Hartung et al. (2010) found three hyperactivity items that were particularly susceptible to halo effects (i.e., Blurts out answers before questions have been completed, Has difficulty awaiting turn, and Interrupts or intrudes on others). The authors did not find any oppositionality items that were uniquely susceptible to halo effects.

Despite the growing body of literature exploring the impact of negative halo effects on symptom ratings of ADHD and comorbid disorders, a review of the literature found no studies examining whether parent raters are susceptible to the same biases as teachers and college students. Parent ratings are considered to be a valuable portion of the assessment process because it provides comparative information in a cost-effective manner that may not be available to the clinician otherwise. Additionally, responses by parents on behavior rating scales are often utilized to guide clinician selection of assessment measures or line of inquiry during clinical interviews (Smith et al. 2009). Thus, it is critical to determine whether parent ratings are also susceptible to negative halo effects and whether the same symptoms are susceptible in parent ratings as those in studies utilizing college student respondents (Hartung et al. 2010).

The Current Study

In the current study, we were interested in determining if the results of previous studies with college student (Hartung et al. 2010; Hartung et al. 2006) and teacher raters (Jackson and King 2004) would generalize to a sample of parent raters. Moreover, we sought to clarify if halo effects would occur when target symptoms were depicted in a variety of vignettes that were consistent with disruptive behavior disorder diagnostic criteria and typical childhood development. For this reason the present study included the vignettes that depicted behaviors consistent with ADHD-Inattentive Type (ADHD-IT), ADHD-Hyperactive/Impulsive Type (ADHD-HT), ADHD-Combined Type (ADHD-CT), ODD, and typical development. We hypothesized that parent raters would also demonstrate the bidirectional halo effect. Specifically, we expected to replicate previous findings by demonstrating the following effects:

  1. a.

    Inattention would be elevated in the ADHD-IT and ADHD-CT vignettes (i.e., clinical effect) as well as the ODD and ADHD-HT Vignettes (i.e., negative halo effect)

  2. b.

    Hyperactivity would be elevated in the ADHD-HT and ADHD-CT vignettes (i.e., clinical effect) as well as the ODD vignette (i.e., negative halo effect) but not the ADHD-IT vignette

  3. c.

    Oppositionality would be elevatedd in ODD Vignette (i.e., clinical effect) as well as the ADHD-CT and ADHD-HT vignettes (i.e., negative halo effect) but not the ADHD-IT vignette

    For any negative halo effects found, we were also interested in determining whether there were any DSM-IV (APA 2000) items that were particularly susceptible to the halo effects based on parent ratings. It was hypothesized that the same items that were particularly susceptible with college student raters (Hartung et al. 2010) would emerge as susceptible with parent raters.

Method

Participants

Study participants were 82 parents (22 fathers, 60 mothers) of 6- to 12-year-olds recruited from three elementary schools in a small town in the Rocky Mountain region of the United States. Parents ranged in age from 25 to 64 (M = 37.15, SD = 7.89). Participants were primarily European American (78 %), with the remaining participants identifying as Hispanic/Latino (2.4 %), Asian American (7.3 %), African American (1.2 %), American Indian (1.2 %), biracial (6.1 %), or other (3.7 %).

Measures

Vignettes

Five vignettes (i.e., ADHD-HT, ADHD-IT, ADHD-CT, ODD, Typical), each with a male and female version were used in the study. Corresponding male and female vignettes were identical other than changes to the first name of the child and the relevant pronouns. Four of the vignettes were modified from those used by Hartung et al. (2010) and the ADHD-HT vignette was developed for this study. The word count for the vignettes ranged from 251 words (i.e., ADHD-IT vignette) to 295 words (i.e., ADHD-CT vignette). Each vignette described an 8-year-old child in the third grade and included target symptoms if applicable. A sample vignette is provided in Appendix A.

The behavior problem vignettes featured Diagnostic and Statistical Manual of Mental Disorders- Fourth Edition (DSM-IV; APA, 1994) symptoms that were either verbatim or paraphrased. It should be noted that although the Diagnostic and Statistical Manual of Mental Disorders- Fifth Edition (DSM-5; APA, 2013) has been published since the time this data was collected, the ADHD symptoms only changed with regard to adding examples that were relevant for older adolescents and adults. The wording of the ADHD symptom criteria did not change from DSM-IV to DSM-5 with the exception of the added parenthetical examples.

Each of the disruptive behavior disorder vignettes (ADHD-CT, ADHD-HT, ADHD-IT, ODD) contained five symptoms respective to each diagnosis depicted. The ADHD-CT vignette contained five symptoms of inattention and five symptoms of hyperactivity/impulsivity. In the typical vignette, the behaviors described were age-appropriate for an 8-year-old and no references were made to symptoms of inattention, hyperactivity, or oppositionality.

Rating Scales

Parent participants used the Disruptive Behavior Rating Scale (DBRS; Barkley and Murphy 1998; Barkley and Murphy 2006) to rate the children in the vignettes. The DBRS includes 26 DSM-IV (APA, 1994) symptoms (i.e., nine ADHD inattention symptoms, nine ADHD hyperactivity symptoms and eight ODD symptoms). Each symptom was rated using a 4-point scale with options of never/rarely, sometimes, often, or very often. Symptoms were considered absent if the participant endorsed never/rarely or sometimes and present if the participant endorsed often or very often. After coding symptoms as absent or present, symptoms were summed to create symptom counts for each dimension based on ratings from each vignette. Given the number of DSM criteria for each behavioral dimension, symptom counts for inattention and hyperactivity ranged from 0 to 9 and symptom counts for oppositionality ranged from 0 to 8. The DBRS has been shown to have good to excellent internal consistency reliability when used with parents (Bauermeister et al. 2005).

Procedure

Study procedures were approved by the university institutional review board and these procedures were in compliance with the ethical standards of the American Psychological Association (APA). Parent participants were recruited through a letter, approved by the school district administration, which was sent home with elementary school students. A brief description of the study and the web address to the online survey were provided in the letter.

The survey was conducted using SurveyMonkey. After providing informed consent on the first page of the survey, participants completed demographic data. In addition to personal demographic data, participants were asked to complete a DBRS regarding one of their own children’s behavior. This was done in an effort to gauge participants’ personal experiences with ADHD and ODD. If a participant had more than one child between the ages of 6- to 12-years of age, he or she was asked to rate the child perceived to have the most difficult behavior. Frequency data indicated that some parents endorsed having a child with clinically significant inattention (37.5 %), hyperactivity (35.4 %), and/or oppositionality (22.5 %) symptoms. These rates probably overestimate the percentage of parents having a child who meets criteria for ADHD or ODD given that other DSM criteria for a diagnosis were not considered (i.e., clinically significant impairment or impairment in two settings).

Next, participants were presented each of the five vignettes one at a time. Vignettes were presented in a counterbalanced order based on symptom presentation and sex of the child depicted (i.e., 2 or 3 vignettes depicting a boy and 2 or 3 vignettes depicting a girl). While the text for the vignette was displayed on-screen, an audio recording of someone reading the vignette was played. The audio-files were included to control for participant reading ability and to increase attention to the vignettes. After reading and listening to a particular vignette and pressing the button to move on, participants rated the child depicted in the vignette. Of note, study materials were presented in a forced response format that required participants to provide an answer to each question prior to continuing on to the next section of the study and incomplete data was omitted from the final data analyses. Following completion of the survey, participants viewed a debriefing statement and were thanked for their participation. For most participants, the survey took 25 to 40 minutes to complete and participants were sent $20 as compensation for their time.

Data Analyses

Symptom counts on each behavioral dimension (i.e., inattention, hyperactivity, and oppositionality) for each of the five vignettes served as dependent variables (DVs). Thus, there were a total of 15 DVs. The data analytic procedure used in the current study was modeled after that used by Hartung et al. (2010). Specifically, halo effects were examined using 12 paired samples t-tests and a Bonferroni correction resulted in an alpha cutoff of p < .004 (.05/12 = .004). Next, item susceptibility to halo effects were examined by conducting 10 pairwise comparisons of proportions for each inattention, hyperactivity and oppositionality symptom using McNemar’s test (Sheskin, 2007). For these analyses, a Bonferroni correction resulted in an alpha cutoff of p < .005 (.05/10 = .005).

Results

Statistical Significance of Halo Effects

To examine whether symptom dimensions were statistically significantly higher in the clinical vignettes than in the Typical vignettes, a series of 12 paired samples t-test were conducted. Based on the Bonferroni correction mentioned previously, an alpha cutoff of p < .004 was used to define significance. The results are shown in Table 1.

Table 1 Comparison of symptoms endorsed in clinical vignettes compared to typical vignette
Full size table

Inattention

It was hypothesized that inattention would be elevated in the ADHD-IT and ADHD-CT vignettes (i.e., clinical effect) as well as the ODD and ADHD-HT vignettes (i.e., negative halo effect). As expected, the inattention symptom counts (target dimensions) were statistically significantly higher in the ADHD-IT and ADHD-HT vignettes than in the Typical vignette (ps < .001). Furthermore, the inattention symptom counts (non-target dimension) were also statistically significant higher in the ADHD-HT and ODD vignettes than in the Typical vignette (ps < .001). Thus, the expected clinical and negative halo effects were documented.

Hyperactivity

It was hypothesized that hyperactivity would be elevated in the ADHD-HT and ADHD-CT vignettes (i.e., clinical effect) as well as the ODD vignette (i.e., negative halo effect) but not the ADHD-IT vignette. As expected, the hyperactivity symptom counts (target dimensions) were statistically significantly higher in the ADHD-HT and ADHD-CT vignettes than in the Typical vignette (ps < .001). Next, the hyperactivity symptom count (non-target dimension) was statistically significantly higher in the ODD vignette than in the Typical vignette (p = .002). Finally, the hyperactivity symptom count (non-target dimension) was not statistically significantly higher in the ADHD-IT vignette than in the Typical vignette (p = .044). Again, the expected clinical and negative halo effects were found.

Oppositionality

It was hypothesized that oppositionality would be elevated in ODD vignette (i.e., clinical effect) as well as the ADHD-CT and ADHD-HT vignettes (i.e., negative halo effects) but not the ADHD-IT vignette. First, the oppositionality symptom count (target dimension) was statistically significantly higher in the ODD vignette than in the Typical vignette (p < .001). Next, the oppositionality symptom count (non-target dimension) was statistically significantly higher in the ADHD-CT vignette than in the Typical vignette (p < .001). However, the oppositionality symptom count (non-target dimension) was not higher in the ADHD-HT or ADHD-IT or vignettes than in the Typical vignette (p = .057 and .033, respectively). Thus, the expected clinical effect was documented but only 1 of 2 expected negative halo effects were documented.

Item Susceptibility to Halo Effects

Given that four non-target dimensions in clinical vignettes were statistically significantly elevated in comparison to the Typical vignette, individual items on these dimensions were examined to determine if any items were particularly susceptible to halo effects. McNemar’s test was used to compare pairs of proportions (Sheskin, 2007). Specifically, two sets of comparisons were conducted to identify susceptible items. First, the rate of endorsement on the non-target dimension of the clinical vignette was compared to the rate of endorsements for the same item in the Typical vignette (e.g., the rate of endorsement of an inattention item in the ODD vignette was compared to the rate of endorsement of the same item in the Typical vignette). Second, the rate of endorsement on the non-target dimension was compared to the rate of endorsement of the same item in the clinical vignettes where the dimension was targeted (e.g., the rate of endorsement of an inattention item in the ODD vignette was compared to the rate of endorsement of the same item in the ADHD-IT and ADHD-CT vignettes).

First, non-target inattention items in the ADHD-HT vignette were examined. There were three inattention items that we examined as possibly susceptible (i.e., Has difficulty sustaining attention in tasks or fun activities; Doesn’t seem to listen when spoken to directly; Avoids, dislikes, or is reluctant to engage in work that requires sustained mental effort; see Table 2). Two of these three items were: (a) endorsed significantly more often in the ADHD-HT vignette (non-target dimension) than in the Typical vignette (non-target dimension); (b) endorsed significantly less often in the ADHD-HT vignette (non-target dimension) than in the ADHD-CT vignette (target dimension); but (c) endorsed as often in the ADHD-HT vignette (non-target dimension) as in the ADHD-IT vignette (target dimension). Thus, given that 2 out of 3 of the above were consistent with a halo effect, these two items (i.e., Difficulty sustaining attention and Doesn’t seem to listen) may be susceptible to being endorsed in the presence of hyperactivity. The third item (i.e., Avoids, dislikes or is reluctant) was: (a) not endorsed significantly more often in the ADHD-HT vignette than in the Typical vignette; (b) endorsed significantly less often in the ADHD-HT vignette than in the ADHD-CT vignette; but (c) endorsed as often in the ADHD-HT vignette as in the ADHD-IT vignette. Given that only 1 out of 3 of the above was consistent with a halo effect, this item does not stand out as particularly susceptible to artificially inflated levels of inattention in the presence of hyperactivity. All other non-target inattention items in the ADHD-HT vignette were endorsed at a rate that was expected (i.e., not distinct from the level of endorsement in the Typical vignette and distinctly lower than the level of endorsement of the same item in the ADHD-CT and ADHD-HT vignettes).

Table 2 Percent of participants who endorsed inattention items across vignettes
Full size table

Second, non-target inattention items in the ODD vignette were examined. Again, there were three inattention items that we examined as possibly susceptible (i.e., Doesn’t seem to listen when spoken to directly; Doesn’t follow through on instructions and fails to finish work; Avoids dislikes, or is reluctant to engage in work that requires sustained mental effort; see Table 2). The Doesn’t seem to listen item was: (a) endorsed significantly more often in the ODD vignette (non-target dimension) than in the Typical vignette (non-target dimension); (b) endorsed significantly less often in the ODD vignette (non-target dimension) than in the ADHD-CT vignette (target dimension); but (c) endorsed as often in the ODD vignette (non-target dimension) as in the ADHD-IT vignette (target dimension). Thus, given that 2 out of 3 of the above was consistent with a halo effect, this item may be susceptible to being endorsed in the presence of hyperactivity. Next, the Doesn’t follow through item was: (a) endorsed significantly more often in the ODD vignette (non-target dimension) than in the Typical vignette (non-target dimension); (b) endorsed significantly less often in the ODD vignette (non-target dimension) than in the ADHD-CT vignette (target dimension); and (c) endorsed significantly less often in the ODD vignette (non-target dimension) than in the ADHD-IT vignette (target dimension). Given that only 1 out of 3 of the above was consistent with a halo effect, this item does not stand out as particularly susceptible to artificially inflated levels of inattention in the presence of oppositionality. Finally, the Avoids, dislikes, and is reluctant item was: (a) not endorsed significantly more often in the ODD vignette (non-target dimension) than in the Typical vignette (non-target dimension); (b) endorsed significantly less often in the ODD vignette (non-target dimension) than in the ADHD-CT vignette (target dimension); but (c) endorsed as often in the ODD vignette (non-target dimension) as in the ADHD-IT vignette (target dimension). Given that only 1 out of 3 of the above was consistent with a halo effect, this item does not stand out as particularly susceptible to artificially inflated levels of inattention in the presence of oppositionality. All other non-target inattention items in the ODD vignette were endorsed at a rate that was expected (i.e., not distinct from the level of endorsement in the Typical vignette and distinctly lower than the level of endorsement of the same item in the ADHD-CT and ADHD-HT vignettes).

Third, non-target hyperactivity items in the ADHD-IT vignette were examined. There was one hyperactivity item that we examined as possibly susceptible (i.e., Has difficulty engaging in leisure activities or doing fun things quietly; see Table 3). This item was: (a) not endorsed more often in the ADHD-IT vignette (non-target dimension) as in the Typical vignette (non-target dimension); (b) endorsed significantly less often in the ADHD-IT vignette (non-target dimension) than in the ADHD-CT vignette (target dimension); but (c) it was endorsed as often in the ADHD-IT vignette (non-target dimension) as in the ADHD-HT vignette (target dimension). However, given that only 1 out of 3 of the above was consistent with a halo effect, this item does not stand out as particularly susceptible to artificially inflated levels of hyperactivity in the presence of inattention. All other non-target hyperactivity items in the ADHD-IT vignette were endorsed at a rate that was expected (i.e., not distinct from the level of endorsement in the Typical vignette and distinctly lower than the level of endorsement of the same item in the ADHD-CT and ADHD-HT vignettes).

Table 3 Percent of Participants Who Endorsed Hyperactivity Items Across Vignettes
Full size table

Next, non-target hyperactivity items in the ODD vignette were examined. There were two hyperactivity items that we examined as possibly susceptible (i.e., Has difficulty engaging in leisure activities or doing fun things quietly and Talks excessively; see Table 3). These two items were: (a) not endorsed significantly more often in the ADHD-IT vignette (non-target dimension) than in the Typical vignette (non-target dimension); (b) endorsed significantly less often in the ADHD-IT vignette (non-target dimension) than in the ADHD-CT vignette (target dimension); but (c) endorsed as often in the ADHD-IT vignette (non-target dimension) as in the ADHD-HT vignette (target dimension). However, given that only 1 out of 3 of the above was consistent with a halo effect, these items do not stand out as particularly susceptible to artificially inflated levels of hyperactivity in the presence of oppositionality. All other non-target hyperactivity items in the ODD vignette were endorsed at a rate that was expected (i.e., not distinct from the level of endorsement in the Typical vignette and distinctly lower than the level of endorsement of the same item in the ODD vignette).

Finally, non-target oppositionality was examined and no items were identified as particularly susceptible (see Table 4). Specifically, all non-target oppositionality items in the ADHD-CT, ADHD-IT and ADHD-HT vignettes were endorsed at a rate that was expected (i.e., not distinct from the level of endorsement in the Typical vignette and distinctly lower than the level of endorsement of the same item in the ODD vignette).

Table 4 Percent of participants who endorsed oppositionality items across vignettes
Full size table

Discussion

In the present study we found that parent ratings of ADHD and ODD symptoms in written vignettes were susceptible to both unidirectional and bidirectional negative halo effects. Parent raters were: (1) more likely to rate a child as inattentive in the presence of hyperactivity symptoms, (2) more likely to rate a child as oppositional when inattention and hyperactivity symptoms were present, and (3) more likely to rate a child as inattentive and hyperactive when oppositionality symptoms were present. Moreover, several specific diagnostic symptoms were found to be particularly susceptible to halo effects. Despite the prevalence of these negative halo effects, parent raters demonstrated fewer negative halo effects compared to teacher (Jackson and King 2004; Stevens et al. 1998; Abikoff et al. 1993) or college student raters (Hartung et al. 2006; Hartung et al. 2010).

Specifically, the negative halo effects and clinical effects of disruptive behavior disorder symptoms that were previously found in college student raters (Hartung et al. 2010) were found to a lesser degree in the current sample of parent raters. Similar to Hartung et al. (2010), bidirectional halo effects were found for ODD and ADHD-CT. That is, children depicted as having ADHD-CT were artificially rated as oppositional and children depicted as having ODD were incorrectly rated as displaying symptoms of inattention or hyperactivity. These findings are also consistent with past research that has examined halo effects in teacher samples (Jackson and King 2004). However, it is important to note that the halo effects observed in the current study appear smaller than those observed in previous studies with teacher and college student raters. This suggests that parents are more sophisticated raters of inattention, hyperactivity and oppositionality in children than teachers or college students. In terms of effect sizes, results from the present study were generally consistent with effect sizes reported in Hartung et al. (2010). Of note, Hartung et al. (2010) reported large effect sizes on inattention and hyperactivity whereas the present study found medium effect sizes (i.e., .35–.52). Agreement between parent and teacher raters is consistently low (Antrop et al. 2002; Wolraich et al. 2004; Tripp et al. 2006; Amador-Campos et al. 2006; Smith et al. 2009); however, this discrepancy in ratings is likely not due to inaccurate and inconsistent ratings. Rather, differences in rater perceptions or variability inherent to child behavior in different settings may offer a more comprehensive explanation for differential ratings (Amador-Campos et al. 2006). Nonetheless, there is little knowledge as to how halo effects specifically may vary across different groups of raters (i.e., mental health professionals, teachers, parents) or if parents or teachers are differentially susceptible to halo effects in their ratings of childhood behavior. To our knowledge this is the first study that strives to address this issue by examining possible halo effects in parents.

A unidirectional halo effect of inattention in the ADHD-HT vignette was also found; illustrating that parents were more likely to describe a child as inattentive if the child displayed hyperactivity. Unlike Hartung et al. (2010), we did not find a unidirectional halo effect of hyperactivity in the ADHD-IT vignette, suggesting that parents do not perceive children as more hyperactive in light of inattention symptoms. However, the inclusion of a hyperactive only type of ADHD vignette is unique to this study and previous research examining halo effects with teachers and college students did not include an ADHD-HT vignette. Therefore, we do not know if this halo effect would also be found for teacher and college student raters.

In addition to examining halo effects of a broader range of symptoms, the present study also examined symptom items to determine if certain items were more susceptible to halo effects than others. In the hyperactivity vignette, two non-target symptoms of inattention were identified as being susceptible to halo effects: Difficulty sustaining attention and Doesn’t seem to listen when spoken to directly. It is possible that parent raters assume that hyperactive behavior inhibits the child’s ability to appropriately attend to others and their environments. Given that previous studies examining negative halo effects in disruptive behavior disorders have not included vignettes or depictions of only hyperactive symptoms, this finding provides useful information regarding the perception of the aforementioned symptoms. Similar to the hyperactivity - only vignette, the inattention symptom Doesn’t seem to listen when spoken to directly was identified as being susceptible to halo effects in the ODD vignette. This finding is consistent with research by Hartung et al. (2010) with college student raters.

Hartung et al. (2010) noted that the word “seem” was omitted from this item in the DBRS (i.e., Doesn’t listen when spoken to directly) even though it is included in the DSM-IV (i.e., Doesn’t seem to listen when spoken to directly) and that this may have contributed to the presence of a negative halo effect on this item because the item may have more overlap with oppositionality when the word “seem” is not included. However, in the present study this item was modified to include the word “seem” and the item was still found to be susceptible to halo effects. Therefore, the deletion of the word “seem” does not appear to responsible for the negative halo effect which suggests that this item may still overlap conceptually with oppositionality when the word “seem” is included. In other words, it is difficult to know if raters are differentiating between children who are not listening due to attention problems or not obeying due to oppositionality.

No other inattention, hyperactivity, or oppositionality items were found to be particularly susceptible to halo effects in the remaining vignettes. Although several additional symptoms were identified as possibly susceptible to halo effects, these symptoms may not have reached the threshold for being susceptible to halo effects for several reasons. First, parent raters might be better skilled at attributing symptoms correctly to the corresponding vignettes. It may also be that variability in parental knowledge and familiarity with disruptive behavior disorders could have influenced the results of this sample. In the current study, only 2 out of 82 parents (2.4 %) reported having an ADHD diagnosis themselves; however, 22.5–37.5 % of parents reported having a child with clinically significant levels of inattention, hyperactivity and/or oppositionality. Although we are not sure what percentage of parents had a child with an ADHD or ODD diagnosis, it does seem that a fair number of parents in our sample have observed a clinically significant number of these symptoms in at least one of their children. Thus, a modest level of familiarity with disruptive behaviors might explain the lower susceptibility to halo effects seen in parents in the current study. In addition, parents likely have more knowledge about child development and behavior than typical college students and this might have also contributed to the lower level of susceptibility to halo effects. Regardless of the reasons for lower susceptibility in parents, these findings illustrate the need for clinicians to be cautious with the language used in the assessment process and to carefully consider the validity of comorbid disruptive behavior disorder diagnoses.

In addition to statistical significance, it is important to consider, the clinical significance of these findings. Although there were several demonstrated halo effects, none of the inflated symptoms counts (oppositionality, hyperactivity, or inattention) met the DSM-5 (APA 2013) suggested cut-off for an ADHD or ODD diagnosis. Thus, halo effects alone would not result in a diagnosis based on parent report. However, if a child was displaying a subthreshold level of symptoms, halo effects might artificially inflate ratings to a clinically significant level. As has been suggested in the literature, clinicians should utilize semi-structured interviews to combat against halo effects that might lead to inaccurate diagnoses (Barkley and Murphy 2006; Hartung et al. 2010).

There are several limitations to this study. First, the sample size is relatively small and homogenous in terms of ethnic identity (78 % European American). Future research should aim to replicate these findings with a larger, more diverse sample of parents. Second, although the present results offer positive findings regarding the accuracy with which parents can identify symptoms, it is important to note that this ability may vary when the caregiver is asked to provide ratings based on his or her own child or a child about whom he or she has more intimate knowledge. Third, the current study depicted discrete behaviors in their purest form as per DSM-IV criteria (APA, 1994). It is unlikely that such discrete behaviors exist in the majority of children with a disruptive behavior disorder, which potentially limits the generalizability of the present findings.

The findings from the present study corroborate past research suggesting that multiple informants and modalities of assessment should be used when diagnosing disruptive behavior disorders. Furthermore, clinicians should consider conducting observations in different settings to provide a more robust conceptualization of the child’s functioning and guard against the influence of negative halo effects. Moreover, future research comparing teacher, parent, and/or clinician ratings in the same study would be helpful in further determining the best standards of assessment for disruptive behavior disorders in children. Future studies may extend this line of research by determining whether or not having a child with a diagnosed disruptive behavior disorder, and presumably increased familiarity with such symptoms, decreases susceptibility to halo effects on rating scales of diagnostic criteria. Such research would likely increase understanding of the limits of rating scale data and provide valuable information to clinicians when making diagnostic decisions.