Attention Deficit/Hyperactivity Disorder (ADHD) is a chronic neurodevelopmental condition typically first diagnosed in childhood, in those who exhibit substantial inattention and/or hyperactivity-impulsivity in a variety of settings (APA 2013). ADHD can manifest as one of three presentations depending on one’s symptoms: predominantly inattentive presentation (ADHD/IA), predominantly hyperactive-impulsive presentation (ADHD/HI), or combined presentation (ADHD/C; APA 2013). Conduct problems is an umbrella term to refer to behavior that is disruptive, external, age-inappropriate, deviates from societal norms, and intrudes on the rights of others (Kimonis et al. 2014). The hyperactive and impulsive symptoms of ADHD in particular have historically been categorized with or alongside conduct problems. Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD) are specific conduct problem diagnoses, and are related to ADHD in several ways. ODD is characterized by an ongoing display of an angry or short-tempered mood and defiance, whereas CD is characterized by more severe conduct problems that breach societal rules, and/or intrude on the rights of others (APA 2013).

ADHD is highly comorbid with ODD and CD, and there is evidence to suggest a developmental progression between these disorders. In fact, individuals with ADHD have a 40 to 60% prevalence rate of ODD, and a 15 to 20% prevalence rate of CD, compared to the general prevalence rates of 3.3% and 4% respectively (Azeredo et al. 2018; Barkley 2015). Ordinarily, these comorbid conduct problems develop after ADHD, making it extremely important for clinicians to be aware of how symptoms develop over time (Barkley 2015). Although a majority of individuals diagnosed with ADHD and ODD do not go on to develop CD, there exists a progressive relationship between symptoms that puts these children at risk for developing CD (APA 2013).

The overlap between ADHD, ODD, and CD could be the result of shared etiologies such as genetics and environment (Faraone 2013; Tuvblad et al. 2008), or potentially flawed research design (Barkley 2015) or biases on the part of raters or diagnosticians. In terms of biases on the part of raters, it is possible that some symptoms of ADHD, ODD, and CD overlap in a way that makes differentiation difficult for non-experts. More specifically, a child with ADHD may not be able to wait patiently in line, whereas a child with ODD may not follow adult directives to stay in line with the intent to be defiant. In fact, this hyperactive/impulsive symptom of ADHD was shown to be commonly mistaken as ODD (Hartung et al. 2010); being unable to wait in line is perhaps viewed as defiant, and thus a symptom of ODD. Moreover, some symptoms of ODD (e.g., arguing with authority figures) and CD (e.g. lying) can result in similarly negative reactions from adults, and non-expert raters may not worry about the subtle distinction between the behaviors. Finally, at times the hyperactive and impulsive behaviors seen in ADHD can result in damage to property, which may cause raters to view a hyperactive or impulsive child as also having symptoms of a more severe conduct problem. Consequently, the similarities between ADHD, ODD, and CD symptoms complicate the diagnostic process.

Evidence-Based Psychological Assessment

Best practices for diagnosing and evaluating children are those that follow evidence-based procedures (Wolraich et al. 2019). Implementing evidence-based, standardized diagnostic practices reduces the probability of over- and under-diagnosis (Pringsheim et al. 2017). A vital component of this practice is assessment, which involves clinicians applying psychometrically sound methods to identify clinical problems. Evidence-based psychological assessment (EBPA) ensures that clinicians take a multi-method and multi-informant approach in which they gather information using a variety of measures and procedures from different sources (Bornstein 2017; Power et al. 2017).

Informant reports, which are ratings/judgements of behavior given by another person, become increasingly important when children are the population of interest, as children might not have the insight nor cognitive capacity to accurately report on their own behavior (De Los Reyes et al. 2015). Clinicians can obtain informant reports in the form of clinical interviews or rating scales. Evidence suggests that interviews and rating scales are correlated (Ramos-Quiroga et al. 2016; Wolraich et al. 2003; Wright et al. 2007). Thus, researchers often rely on rating scales because they are a cheaper and time efficient alternative to interviews. Additionally, rating scales provide clinicians and researchers with a norm-referenced score to which they can easily compare a target child’s behavior (McMahon and Frick 2005). Thus, it is common to conduct a clinical interview with one informant, but rely on rating scales to gather information from multiple informants. Informant reports provide clinicians with essential information needed to diagnose a child with ADHD, ODD, or CD. While teacher-reports are important, parents/guardians offer a unique perspective because they are typically the only individuals to observe the child’s behavior over years (Kazdin 2005). Accordingly, clinicians rely heavily on parent reports during the diagnostic process and could weigh their ratings more strongly than other informant reports. Moreover, parents act as gatekeepers to a child receiving an EBPA because their evaluation of their child’s behavior will determine whether or not they seek professional help. Thus, it is crucial that parents can accurately appraise and report on their child’s behavior; and that researchers and clinicians understand the ways that parents make their ratings.

Negative Halo Effects

Researchers have found that raters can accurately distinguish broadly between clinical and non-clinical groups of children (American Academy of Pediatrics [AAP], 2000), via rating forms. However, a critical matter is whether or not these raters can accurately distinguish between different diagnostic categories such as ADHD, ODD, and CD. One factor that might prevent this differentiation between diagnostic categories is referred to as a negative halo effect (also referred to as a horn effect). Halo effects occur when an individual’s impression of one characteristic leads to a positive (halo) or negative (negative halo) rating of an unrelated characteristic (Thorndike 1920; Nisbett and Wilson 1977). Negative halo effects in mental health ratings arise when an individual erroneously rates a child exhibiting one behavior (e.g., oppositionality) as exhibiting another behavior (e.g., hyperactivity; Hartung et al. 2006). Unidirectional negative halo effects occur when one behavior (e.g., oppositionality) leads to the erroneous rating of a secondary behavior (e.g., hyperactivity), but the secondary behavior does not lead to erroneous ratings of the primary behavior. Bidirectional negative halo effects occur when one behavior (e.g., oppositionality) leads to the erroneous rating of a secondary behavior (e.g., hyperactivity) and the secondary behavior also leads to the erroneous rating of the primary behavior (Hartung et al. 2010).

Negative halo effects have important conceptual implications for the assessment of behavior problems. A negative halo effect is essentially a cognitive error or bias wherein a person judges a target more harshly because of previous negative information (Hartung et al. 2010). For example, if you learn that an adolescent has shoplifted in the past, you might view any subsequent behavior, even ambiguous or relatively minor misbehavior, more harshly. Thus, it is important to understand this cognitive error, as well as the direction of the error, so mental health professionals can avoid mis- and overdiagnosis.

There is a history of research examining negative halo effects in ratings of ADHD and oppositional/defiant symptoms. The initial three studies in the 1980s and 90s found unidirectional negative halo effects such that teachers who observed a boy depicting oppositionality (in person or on video tape) erroneously endorsed symptoms of hyperactivity (Abikoff et al. 1993; Schachar et al. 1986; Stevens et al. 1998). Further, Jackson and King (2004) expanded the research on teacher ratings of children with ADHD and ODD to examine gender differences by comparing ratings of girls and boys after viewing a video tape. Researchers found bidirectional negative halo effects in that the presentation of ODD symptoms led teachers to erroneously endorse symptoms of ADHD, and the presentation of ADHD symptoms led teachers to erroneously endorse symptoms of ODD (Jackson and King 2004). However, their findings suggest that these bidirectional effects differed across genders. Specially, teachers rated boys presenting with ODD symptoms as having ADHD more often than they rated girls presenting with ODD symptoms as having ADHD. Conversely, teachers rated girls presenting with ADHD symptoms as having ODD more often than they rated boys presenting with ADHD symptoms as having ODD (Jackson and King 2004). All results had medium effect sizes.

To continue this line of research, three additional studies were conducted since 2006 with written vignettes of children depicting ADHD and/or ODD behaviors. First, a study was conducted to investigate negative halo effects in undergraduate college student ratings of children with ADHD and ODD (Hartung et al. 2006). The results suggested that bidirectional negative halo effects exist for ratings of both boys and girls, again with a medium effect size. Thus, both college student and teacher ratings of children with ADHD and ODD show bidirectional negative halo effects. Subsequently, Hartung et al. (2010) further investigated college students’ ratings of children with ADHD and ODD to detect whether specific individual symptoms are prone to halo effects. Findings from this study support past evidence of a bidirectional negative halo effect between ADHD and ODD. However, the negative halo effect was stronger between ADHD/C and ODD (d = .63; a medium effect size) than it was between ADHD/IA and ODD (d = .14; a small effect size).

More recently, researchers investigated parent ratings of children with ADHD and ODD and examined specific symptomatology vulnerable to halo effects (DeVries et al. 2017). Researchers found bidirectional negative halo effects in parent ratings of ODD and ADHD/C (with small effect sizes), which is consistent with previous studies (DeVries et al. 2017). It is important to note that this was the first study to use an ADHD/HI vignette to directly examine the hyperactive symptoms specifically. The researchers reported a unidirectional negative halo within ADHD subtypes, such that a child described as being hyperactive and impulsive were erroneously rated as being inattentive (DeVries et al. 2017).

Current Study

Although various researchers have documented the presence of negative halo effects in ratings of ADHD and ODD, there is a gap in the literature in terms of parent ratings of CD. This is important because of the comorbidity (Azeredo et al. 2018), developmental progression (APA 2013), and shared etiology (Faraone 2013) between CD and these other disorders. It can be difficult for non-expert raters such as parents to distinguish between these overlapping symptoms when completing behavioral rating scales, especially the hyperactive/impulsive symptoms of ADHD, and the symptoms of ODD and CD. Evidence-based psychological assessment relies on these parent ratings, so understanding when parents might display negative halo effects is essential. Therefore, this study examined whether the presence of negative halo effects in parent ratings of ADHD/HI and ODD also extend to CD.

Specifically, the current study was a true experiment in which parents were randomly assigned to read a vignette depicting a boy with symptoms of ADHD/HI, ODD, or CD. Additionally, a fourth vignette that depicted a typical child was included to serve as the comparison condition. The purpose of this study was to identify possible negative halo effects in parent ratings of boys with ADHD/HI, ODD, and CD. Three primary hypotheses and an exploratory hypothesis of mother versus father ratings were developed to examine possible bias in parent ratings of boys with ADHD/HI, ODD, and CD.

Hypotheses

Overall, our hypotheses included bidirectional negative halo effects between all parent ratings of ADHD/HI, ODD, and CD symptoms. These were tested with three ANOVAs as outlined below, and although each ANOVA only tested for unidirectional negative halo effects, combining the results indicated whether or not bidirectional negative halo effects were detected. We were also interested in mother ratings versus father ratings, given that mothers and fathers have been shown to rate child behaviors differently (Duhig et al. 2000; Sanders et al. 2007), and in general it is important to study potential sex differences (Hartung and Lefler 2019).

Hypothesis 1. It was hypothesized that parents would erroneously rate the boy in the ADHD/HI condition as presenting with more symptoms of ODD and CD than the boy in the typical condition, as measured by the Rating Scale for Disruptive Behavior Disorders and analyzed using ANOVA. This would demonstrate a negative halo effect such that when a parent reads about a child presenting with ADHD/HI symptoms, they erroneously rate the child as presenting with elevated ODD and/or CD symptoms.

Hypothesis 2. It was hypothesized that parents would erroneously rate the boy in the ODD condition as presenting with more symptoms of ADHD/HI and CD than the boy in the typical condition, as measured by the Rating Scale for Disruptive Behavior Disorders and analyzed using ANOVA. This would demonstrate a negative halo effect such that when a parent reads about a child presenting with ODD symptoms, they erroneously rate the child as presenting with elevated ADHD/HI and/or CD symptoms.

Hypothesis 3. It was hypothesized that parents would erroneously rate the boy in the CD condition as presenting with more symptoms of ADHD/HI and ODD than the boy in the typical condition, as measured by the Rating Scale for Disruptive Behavior Disorders and analyzed using ANOVA. This would demonstrate a negative halo effect such that when a parent reads about a child presenting with CD symptoms, they rate erroneously the child as presenting with elevated ADHD/HI and/or ODD symptoms.

Method

Participants

Participants included a sample of 391 parents/guardians (mothers, n = 236; fathers, n = 126; guardians, n = 11; other, n = 2) ranging in age from 18 to 74 years (M = 43.48, SD = 8.75). Approximately 97% of participants had children ranging in age from infancy to emerging adulthood, whereas 3% of participants had adult children (M child age = 11.87 years). Participants predominantly identified as White/Caucaisan (73.5%, n = 280) and female (63.7%, n = 249). The remaining participants identified as African American or Black (3.4%, n = 13), Asian (2.1%, n = 8), Hispanic/Latino (8.7%, n = 33), Native American or Alaska Native (6.8%, n = 26), Native Hawaiian or Pacific Islander (.80%, n = 3), multi-racial (2.9%, n = 11), or did not respond. In terms of educational level, participants reported having some high school (1%, n = 4), a high school diploma (3.7%, n = 14), some college but no degree (11.3%, n = 43), an associate’s degree (8.4%, n = 32), a bachelor’s degree (44.9%, n = 171), a master’s degree (14.2%, n = 54), a professional degree (9.5%, n = 37), a doctorate degree (6.8%, n = 26), or did not respond. Participants’ yearly income was reported to be $30,000 or less (10.5%, n = 40), $30,000 to $50,000 (23.9%, n = 91), $50,000 to $100,000 (41.3%, n = 157), $100,00 to $150,000 (16.8%, n = 64), more than $150,000 (7.4%, n = 28), or did not respond. Finally, the number of children per participant ranged from one to seven (M = 1.52), but a majority of participants reported only having one child (66.3%, n = 250). The authors recruited a total of 502 participants, but after data cleaning procedures, 111 participants were excluded (for reasons explained below), and then analyses were conducted with the remaining 391 participants.

Procedure

IRB approval was obtained prior to data collection, and the study was preregistered using the Open Science Framework (OSF). Parents were recruited to be participants via three methods. First, participants with children in New York and Iowa public and parochial schools were recruited through the school’s Parent Teacher Organization/Association (PTO/A) President, and contacted via email. New York and Iowa were chosen due to connections the first author has in those states. Second, participants were recruited online via social media. Specifically, the author posted the survey link on two Facebook groups. Additionally, acquaintances of the authors were asked to post the survey link on their personal social media pages. Finally, a snowball sampling method (Baltar and Brunet 2012) was implemented in which all participants from recruitment methods one and two were asked to send the survey to other parents/guardians in their circles.

The survey was hosted on Qualtrics, which directed all participants first to an online informed consent document. Participants were then randomly assigned to one of the four conditions, all of which described an 11-year-old boy. After reading their assigned vignette, participants rated the boy’s behavior. They were not able to go back and re-read the vignette after they were directed to the page with questions. Next, participants were asked to complete a demographics form. Finally, participants were compensated via either a $5 gift card to Dunkin’ Donuts® or Amazon®.

Vignettes

Participants read a vignette that portrayed a boy with either ADHD/HI, ODD, CD, or typical behavior (Table 1). The ADHD/HI and ODD vignettes were adapted from the Hartung et al. 2006 study. All four vignettes ranged from 171 to 173 words and describe a boy named “Sam” because it was among the most popular boy names during the years that current elementary-school aged children were born (Social Security Administration 2018). The three experimental vignettes all included eight symptoms of the respective disorder. Both the ODD and CD vignettes depicted relatively mild symptoms, because more severe symptoms would be easier to differentiate from ADHD. For example, one severe symptom of CD is “Has forced someone into sexual activity” (APA 2013, p. 470), which is not as likely to be misperceived as ADHD/HI. On the other hand, a milder symptom such as stealing (APA 2013) might be confused with ADHD/HI or ODD. The typical vignette describes a boy who exhibits developmentally appropriate behavior to serve as a comparison condition.

Table 1 Vignettes
Full size table

All four vignettes featured a boy to correspond with the progression of the past research conducted on negative halo effects. More specifically, the first three studies to examine negative halo effects in ratings of ADHD and ODD had teachers rate only boys (Abikoff et al. 1993; Schachar et al. 1986; Stevens et al. 1998). Notably, ADHD and ODD are more prevalent among boys than girls (APA 2013), which is likely the reason these initial studies only included boys. Girls were added to vignettes after a series of studies found consistent evidence of the presence of unidirectional negative halo effects in teachers’ ratings of ADHD and ODD in boys. Thus, because CD was added to extend this line of research, a return to a focus on male-only vignettes is appropriate.

Measures

Rating Scale for Disruptive Behavior Disorders

(RS-DBD; Silva et al. 2005). The parent version of the Rating Scale for Disruptive Behavior Disorders is a 41-item measure that was created to assess parent ratings of ADHD, ODD, and CD from the DSM-IV. The RS-DBD contains 18 DSM items associated with ADHD, 8 DSM items associated with ODD, and 15 DSM items associated with CD (Silva et al. 2005). Given that the current study focuses only on the ADHD/HI presentation, the 9-items pertaining to the inattentive symptoms of ADHD were not included. Thus, participants were given a 32-item subscale of the RS-DBD that includes questions pertaining to hyperactivity, oppositionality, and conduct behaviors. Parents were asked to rate each item on a 4-point frequency scale (0 = not at all, 1 = just a little, 2 = pretty much, 3 = very much). Sample items include: “Does not seem to listen when spoken to directly,” “Loses temper,” and “Deliberately annoys people.” In the current study, the reliability for the ADHD/HI and ODD subscales were good (ADHD, α = .87; ODD, α = .86), and the reliability for the CD subscale and the overall scale were excellent (CD, α = .91; Overall scale, α = .92).

Demographics Form

The demographics questionnaire, created by the authors, asked participants to disclose their sex, age, race/ethnicity, parental status (e.g., father/mother/guardian), educational level, age of children, income, occupation, and marital status.

Attention Checks

Two attention checks were included in the RS-DBD to assess participants’ attention to the items they are being asked. Both items occurred within the RS-DBD, and directed participants to select specific anchors (i.e., “very much” or “just a little”). This was to guard against random responding. Participants were excluded from the analyses if they failed both attention checks.

Results

Preliminary Analyses

Data Preparation

An a priori power analysis using G*Power version 3 indicated that a sample of 304 participants (approximately 76 participants per group; Faul et al. 2007) was required to detect a medium effect size (per Hartung et al. 2006). A total of 502 participants completed the study. Per our a priori rules, four participants were excluded because their IP addresses were from outside of the United States, and another 64 participants were excluded because they did not complete a minimum of 80% of the survey. Lastly, an additional 40 participants were excluded for completing the study an unreasonable amount of time (i.e., in less than 2 min). After this initial cleaning was complete, none of the remaining participants failed both attention checks. However, scatterplots of data revealed three extreme outliers in the number of CD symptoms (i.e., 15 of 15 symptoms) endorsed by participants who read the typical vignette. This implausible endorsement rate (i.e., endorsing severe levels of conduct in a typical child) indicated poor attention, and thus these participants were excluded. Consequently, a total of 391 participants were included in the analyses. All assumptions for ANOVAs were met. Finally, effect size for ANOVAs is referred to as η2 and can be categorized as small (η2 = .01), medium (η2 = .06), and large (η2 = .14; Perugini et al. 2018).

Symptoms Endorsed

In addition, the mean number of symptoms endorsed in each vignette condition were examined (Table 2). As can be seen in the table, participants who read the ADHD/HI vignette did not endorse enough symptoms of any disorder to meet a diagnostic cutoff (i.e., 6 or more symptoms for ADHD, 4 or more symptoms for ODD, or 3 or more symptoms for CD per DSM-5). However, participants who read the ODD vignette endorsed sufficient symptoms for the ODD diagnostic cutoff, and participants who read the CD vignette endorsed sufficient symptoms to meet the CD diagnostic cutoff.

Table 2 Mean Number of ADHD/HI, ODD, and CD Symptoms Endorsed across Vignettes
Full size table

Primary Analyses

Three ANOVAs were conducted to compare parents’ behavior ratings. Specifically, analyses were used to examine parents’ endorsement of ADHD/HI, ODD, and CD symptoms between vignette conditions. Parents’ endorsement of symptoms was measured using the Rating Scale for Disruptive Behavior Disorders. Specifically, a symptom was considered endorsed if parents rated the boy in the vignette as displaying it “Pretty much” or “Very much.” Three one-way between subjects ANOVAs revealed statistically significant differences between conditions in ratings of ADHD/HI symptoms, F (3, 387) = 32.74, p < .001, η2 = .20; ratings of ODD symptoms, F (3, 387) = 27.75, p < .001, η2 = .18; and ratings of CD symptoms, F (3, 387) = 9.17, p < .001, η2 = .07. Next, post hoc comparisons using the Tukey HSD (“honestly significant difference”) test and post hoc t-tests were conducted to evaluate specific hypotheses (Table 3).

Hypothesis 1 (ADHD Vignette). It was hypothesized that parents would erroneously rate the boy in the ADHD/HI vignette as presenting with more symptoms of ODD and CD than the boy in the typical vignette. The first post hoc comparison using Tukey’s HSD test indicated that parents who read the ADHD/HI vignette endorsed equivalent symptoms of ODD (M = 2.07, SD = 2.17) as parents who read the typical vignette (M = 1.56, SD = 2.23; p = .385). The second post hoc Tukey’s test indicated that parents who read the ADHD/HI vignette endorsed equivalent symptoms of CD (M = 2.56, SD = 3.46) as parents who read the typical vignette (M = 2.03, SD = 3.45; p = .526). Thus, this hypothesis was not supported. A negative halo effect in which a parent reads about a child presenting with ADHD/HI symptoms and erroneously rates the child as presenting with ODD or CD symptoms was not demonstrated (Table 3).

Hypothesis 2 (ODD Vignette). Next, it was hypothesized that parents would erroneously rate the boy in the ODD vignette as presenting with more symptoms of ADHD/HI and CD than the boy in the typical vignette. The first post hoc Tukey’s test in this ANOVA indicated that parents who read the ODD vignette endorsed significantly more symptoms of ADHD/HI (M = 2.61, SD = 2.35) than parents who read the typical vignette (M = 1.42, SD = 1.97; p = .003, d = .03). Thus, this portion of the hypothesis was supported, demonstrating a negative halo effect such that when a parent reads about a child presenting with ODD symptoms, they erroneously rated the child as presenting with elevated ADHD/HI symptoms, with a small effect size (Table 3). The second Tukey’s test indicated that parents who read the ODD vignette endorsed equivalent symptoms of CD (M = 2.97, SD = 3.42) as parents who read the typical vignette (M = 2.03, SD = 3.45; p = .938). Thus, this portion of the hypothesis was not supported. A negative halo effect in which a parent reads about a child presenting with ODD symptoms and erroneously rates the child as presenting with CD symptoms was not demonstrated (Table 3).

Hypothesis 3 (CD Vignette). Finally, it was hypothesized that parents would erroneously rate the boy in the CD vignette as presenting with more symptoms of ADHD/HI and ODD than the boy in the typical vignette. The first Tukey’s test for this ANOVA indicated that parents who read the CD vignette endorsed significantly more symptoms of ADHD/HI (M = 2.35, SD = 2.29) than parents who read the typical vignette (M = 1.42, SD = 1.97; p = .031, d = .02). The second Tukey’s test indicated that parents who read the CD vignette endorsed significantly more symptoms of ODD (M = 3.06, SD = 2.43) than parents who read the typical vignette (M = 1.56, SD = 2.23; p < .001, d = .03). Thus, this hypothesis was supported, demonstrating a negative halo effect such that when a parent reads about a child presenting with CD symptoms, they erroneously rated the child as presenting with elevated ADHD/HI and ODD symptoms, with a medium effect size (Table 3).

Table 3 Tukey HSD Post Hoc Analyses Comparing Symptomatic Vignettes to Typical Vignette for All Parents
Full size table

Mother Only Analyses

In addition to the primary analyses which included the full sample, we were also interested in whether mothers and fathers differed in their ratings of the vignettes. However, because there were so few fathers across the vignette conditions (n = 126), and because of some suspected random responding by these father participants, we were only able to explore mother ratings in depth. Thus, the following data represent the same ANOVAs and Tukey’s post hoc tests as the primary hypotheses, but for mother ratings only (n = 236).

Three one-way between subjects ANOVAs revealed statistically significant differences between conditions in mother ratings of ADHD/HI symptoms, F (3, 232) = 27.00, p < .001, η2 = .26; ratings of ODD symptoms, F (3, 232) = 27.04, p < .001, η2 = .26; and ratings of CD symptoms, F (3, 232) = 17.69, p < .001, η2 = .19. Looking more closely at the Tukey’s post hoc results, the mother only results were identical to the full sample, but with one additional negative halo effect in the ODD vignette condition (Table 4). For the first ANOVA, hypothesis 1 (the ADHD vignette) with mother ratings only, there were no significant negative halo effects (p = .560 for ODD; p = .835 for CD) which is in line with the full sample.

Table 4 Tukey HSD Post Hoc Analyses Comparing Mother Ratings of Symptomatic Vignettes to Typical Vignette
Full size table

Second, for hypothesis 2 (the ODD vignette) with mother ratings only, two negative halo effects were detected, as opposed to just one that was found with the full sample. Specifically, mothers who read the ODD vignette endorsed significantly more symptoms of ADHD/HI (M = 2.39, SD = 2.40) than mothers who read the typical vignette (M = 1.02, SD = 1.81; p = .015, d = .57). Likewise, mothers who read the ODD vignette endorsed significantly more symptoms of CD (M = 2.59, SD = 3.21) than mothers who read the typical vignette (M = 1.14, SD = 2.64; p = .045, d = .50, medium effect sizes).

Finally, for hypothesis 3 (the CD vignette) with mother ratings only, two negative halo effects were detected, mirroring the data with the full sample. Specifically, mothers who read the CD vignette endorsed significantly more symptoms of ADHD/HI (M = 2.43, SD = 2.34) than mothers who read the typical vignette (M = 1.02, SD = 1.81; p = .008, d = .59, medium effect size). Likewise, mothers who read the CD vignette endorsed significantly more symptoms of ODD (M = 3.25, SD = 2.33) than mothers who read the typical vignette (M = 1.00, SD = 1.79; p < .001, d = 1.07, large effect size).

Discussion

Researchers have identified the presence of negative halo effects between ADHD and ODD in the past; however, these studies have mostly used teacher and college student raters, and have not included CD (Abikoff et al. 1993; Hartung et al. 2006; Hartung et al. 2010; Jackson and King 2004; Schachar et al. 1986; Stevens et al. 1998). This study aimed to extend the literature by examining the presence of negative halo effects in parent ratings of Conduct Disorder in addition to ADHD/HI and ODD. It was hypothesized that there would be bidirectional negative halo effects between all three disorders. Altogether, results from the present study suggest that parent ratings of ADHD/HI, ODD, and CD symptoms in written vignettes are susceptible to some particular “downward” negative halo effects. Specifically, our data suggest a negative halo effect from the relatively more severe disorder downward to the relatively less severe disorder. That is, when CD is described, both ODD and ADHD/HI were erroneously endorsed; when ODD was described, ADHD/HI was erroneously endorsed; but when ADHD/HI was described, neither ODD nor CD were endorsed. For mother raters only, this general trend was violated in only one analysis (i.e., for mothers only, when ODD was described, CD was erroneously endorsed; but all other mother-only analyses were consistent with the full sample). Thus, in the current study, unidirectional, downward negative halo effects were our most consistent finding.

More specifically, although a bidirectional negative halo was predicted between ADHD and ODD, only a unidirectional negative halo effect was found. Indeed, as stated above, when parents (and mothers only) read about a boy presenting with ODD symptoms, they erroneously rated the boy as presenting with the relatively less severe ADHD/HI symptoms, but not the other way around. Interestingly, this finding is consistent with the three earliest studies investigating negative halo effects in teacher ratings of ADHD and ODD (Abikoff et al. 1993; Schachar et al. 1986; Stevens et al. 1998), but not with the four most recent studies which found bidirectional negative halo effects between in teacher ratings (Jackson and King 2004), college student ratings (Hartung et al. 2010; Hartung et al. 2006) and parent ratings (DeVries et al. 2017) of ADHD and ODD. This is curious given that the methodology of these early studies differs most from the current methodology (i.e., live or video-taped children vs. vignettes; teachers vs. parents). However, this demonstrates that downward negative halo effects have been found consistently over decades, across raters, and via various modalities.

Likewise, despite the predicted bidirectional halo effect between ADHD and CD, a unidirectional negative halo effect downward from CD to ADHD/HI was demonstrated for all parents and mothers raters only. Thus, raters did not endorse CD symptoms when only ADHD/HI was described. This suggests that the relatively less severe ADHD/HI does not erroneously inflate CD ratings in boys. Regardless, the finding that the presence of CD symptoms artificially inflates ratings of ADHD/HI symptoms is notable and novel.

Third, despite the prediction of a bidirectional halo effect between ODD and CD, only a downward unidirectional negative halo effect was demonstrated in our analyses with all parents. However, for mother raters only, the bidirectional halo was found. In particular, in the analysis with all parents, the boy in the CD vignette was rated as having symptoms of oppositionality but not the reverse; but for mother raters only there was a bidirectional negative halo between CD and ODD. This is a novel finding that, to the authors’ knowledge, has not been examined in past studies. That is, conduct disorder can and should be included in the discussion of negative halo effects in ADHD and ODD.

In all, the current findings might suggest that parents are more willing to endorse symptoms of a relatively less severe disorder (i.e., ADHD) when a boy is presenting symptoms of a relatively more severe disorder (i.e., ODD), rather than the reverse. Indeed, it is possible that parents assume the presence of a more severe conduct disorder presupposes the existence of a less severe behavioral issue. It follows that the more negative an initial behavior, the larger or stronger the subsequent negative halo.

On the other hand, the current findings could be due to parents’ poor appraisal of negative behaviors overall. For example, researchers who examined household survey data of over 4000 parents found that 9% of parents endorsed enough symptoms for their child to meet the diagnostic criteria for ODD (Sanders et al. 2007), despite the 3.3% prevalence rate of ODD noted in the DSM-5(APA 2013). This elevated endorsement of ODD could be due to misperception. Parents have difficulty judging and recognizing their child’s problem behavior, compounded by the fact that evidence-based resources are not readily accessible to help guide them (Johnston and Burke 2019).

Finally, we should note that the number of non-target symptoms that parents endorsed for each vignette was examined to determine clinical impact. In order to meet the diagnostic threshold for ADHD/HI, ODD, and CD parents would have to endorse 6 symptoms of hyperactivity/impulsivity, 4 symptoms of oppositionality, and 3 symptoms of conduct respectively (APA 2013). The negative halo effects that were detected in the current study did not meet or surpass the clinical threshold for a diagnosis (Table 1), which is consistent will all previous negative halo effect studies (Abikoff et al. 1993; DeVries et al. 2017; Hartung et al. 2006; Hartung et al. 2010; Jackson and King 2004; Schachar et al. 1986; Stevens et al. 1998). This means that negative halos in isolation would likely not lead to a diagnosis in a child who is completely asymptomatic, but could lead to an incorrect diagnosis for a child who presents with a subthreshold number of hyperactive or oppositional symptoms.

Altogether, the current findings suggest that negative halo effects alone would not lead to misdiagnosis unless a child presents with a subthreshold number of symptoms of ADHD/HI or ODD, and has symptoms of a relatively more severe disorder. Overall, the current findings demonstrate a pattern of flawed judgment in parent raters that could lead to impactful clinical errors, especially for relatively less severe disorders. In terms of effect size, the negative halo effect detected in the current study ranged from small to large. Previous studies have found both small (Abikoff et al. 1993; DeVries et al. 2017; Hartung et al. 2010; Stevens et al. 1998) and medium effect sizes (DeVries et al. 2017; Hartung et al. 2006; Hartung et al. 2010; Jackson and King 2004; Schachar et al. 1986).

Clinical Implications

The results from the current study have implications for evidence based psychological assessment. As mentioned previously, clinicians rely on parent report as a diagnostic tool during an EBPA with child and adolescent clients (Bornstein 2017; Handler and DuPaul 2005). In particular, when assessing for disruptive behavior disorders, Connor et al. (2010) argue that ODD symptoms should be assessed routinely in ADHD evaluations. Thus, the findings from the current study reinforce the notion that clinicians should understand that when assessing the disruptive behavior disorders, parents have a tendency to inflate ratings of the less severe disorder when there is evidence of a more severe disorder. Clinicians should be aware of this downward negative halo effect to avoid misdiagnosis. When children are correctly diagnosed, then they can receive treatment that targets their specific problem behaviors. For example, stimulant medication has been shown to be an effective, first-line treatment for children with ADHD (Faraone et al. 2006) and ADHD with comorbid ODD or CD (Pringsheim et al. 2015), along with behavioral parent training and classroom interventions (Evans et al. 2018). On the other hand, Multisystemic Therapy (MST) has been shown to be a well-established treatment for CD (McCart and Sheidow 2016). Therefore, carefully distinguishing ADHD from these relatively more severe disorders could help improve the specificity of our treatment recommendations.

Both rating scales and clinical interviews as part of EBPA are implicated here. For example, if a parent’s rating scale suggests that the child exhibits high levels of both hyperactive/impulsive and oppositional behavior, the clinician may want to inquire further about the specific context in which each individual symptom is said to occur. Moreover, an attempt should be made to help parents focus on the specific content of each symptom, rather than an overall vague sense of “misbehavior.” Being aware of these negative halo effects in a clinical interview may be helpful in giving context and nuance to the symptoms a parent endorsed on a rating scale.

Lastly, the multi-method and multi-informant approach of EBPA ensures that clinicians make diagnostic decisions that are informed by numerous individuals and pieces of evidence rather than just using one diagnostic tool or one reporter, which serves to increase accuracy. In particular, the multi-method approach can help improve diagnostic accuracy when clients present with subthreshold symptoms because even if one parent demonstrates negative halo effects, clinicians can incorporate other information - such as teacher ratings and behavioral observations - when making a diagnostic decision to combat the impact of rater bias. In fact, mothers are usually the individuals who bring a child in for a diagnostic assessment (Power et al. 2001; Sanders et al. 2007; Wright et al. 2007). Thus, gathering data from other adults in the child’s life is paramount, as any one parent might display a negative halo effect.

Limitations and Future Directions

There were several limitations in the current study that should be considered when interpreting the results. First, this study was an analogue study and may not replicate if these parents were to complete rating forms for their own child. Participants in this study rated a fictional child from a vignette with whom they had no prior experience. When parents complete rating forms on their own child’s behavior, they make judgements based on the child’s behavior in numerous settings and over a longer period of time. Hence, the clinical significance is not entirely clear because we do not know whether the results will generalize beyond vignettes or with parents rating their own children. Future studies should investigate the presence of negative halo effects in parent ratings of their own children.

Second, the current study used vignettes that described a boy, and the participants were mostly mothers. Consequently, it is unknown whether the results would generalize to parent ratings of a girl presenting with the same symptoms, or what might change with more father data. We elected to use boy vignettes to control for the sex of the child, and because in introducing CD to the ADHD halo effects literature we had to pick a starting point. Nonetheless, it is important for future research teams to examine girl vignettes as well, especially given that one research team found that a child’s sex influenced diagnostic decisions in that therapists were more likely to diagnose the subthreshold boy vignette with ADHD than the subthreshold girl vignette (Bruchmüller et al. 2012).

Third, the parents in our study were 73.5% White. Because of this, our results may only be applicable to White families, and may have limited utility for Black, Indeginouis, or other People of Color. In addition to this, we chose the name “Sam” because it was a popular name in the correct era. However, this name may have been read as White by the participants, so nothing can be said about potential racial differences that might impact or cause negative halo effects. Importantly, Baglivio et al. (2017) found that Black children were 40–54% more likely to be diagnosed with CD than White children despite comparable behavior. Future research examining negative halo effects between ADHD, ODD, and CD can help determine whether parents and/or clinicians display negative halo effects during assessments of children, particularly with regard to race. These limitations should certainly be considered in future studies of negative halo effects in ADHD.

Overall Conclusions

Overall, the current study revealed the presence of negative halo effects in parent ratings of ADHD/HI, ODD, and CD in vignettes featuring 11-year-old boys. The general trend we found was that the presence of a relatively more severe disorder artificially inflated ratings of the relatively less severe disorder (e.g., a boy described as having CD was also rated as having the less severe symptoms of ADHD/HI and ODD). These significant negative halo effects largely replicate past work in the area, and, in a notable step forward, extend the findings to conduct disorder. Our data reemphasizes the importance of EBPA throughout the diagnostic process to decrease the chance of misdiagnosis.