Introduction

Despite increases in public awareness of mental health issues, there exists an unmet mental health need among children with a substantial proportion of those in need reporting they have never received care (Merikangas et al., 2011). Further disparities emerge in the receipt of services between individuals experiencing internalizing and externalizing issues in school and when considering community-based services (Bradshaw, Buckley, & Ialongo, 2008; Splett et al., 2018). In fact, in a sample of school referred youth, students with less observable internalizing concerns were significantly less likely to report school- and community-based intervention services than students with externalizing concerns, and no more likely than students with internalizing and externalizing behaviors rated in the normal range (Splett et al., 2018).

Concerns regarding unmet need and disparities in service receipt have resulted in several calls for improved practice, including teacher training in children’s mental health issues, universal mental health screening, and increased implementation of evidence-based practices (Weist et al., 2018). Identifying youth with mental health concerns and connecting them to practices that work is important because poor mental health is related to notably lower educational achievements, substance abuse, violence, compromised health, and reduced life satisfaction (Gilman & Huebner, 2006; Patel, Flisher, Hetrick & McGorry, 2007). Further, if untreated, youth experiencing internalizing behavior problems are at greater risk of impaired school functioning, interpersonal and familial conflicts, and an increased possibility of engaging in risky behaviors, including suicide (Aseltine, Gore, & Gordon, 2000; Hawton, Saunders, & O’Connor, 2012; McWhirter & Page, 1999). Similarly, youth experiencing externalizing problems are at risk of poor academic performance, reduced school attendance, and greater emotional difficulties (Davis, Kreczek, & Mcintosh, 2006), as well as placement in special education programs (Wagner, 1995; Bradshaw et al., 2008) and later school failure (Wagner, 1995).

The purpose of this paper is to examine teachers’ identification of, concern for, and response to externalizing and internalizing behavior problems to better inform the content and design of teacher training efforts. This is critical given that most previous efforts have not resulted in desired improvements and the disparity between identification and treatment of internalizing and externalizing behavior problems continues to exist (Moor et al., 2007; Powers, Wegmann, Blackman, & Swick, 2013; Splett et al., 2018). In the sections that follow, we further examine the need for and current status of teacher training in the identification and treatment of internalizing and externalizing behavior problems and then describe the role of teachers in identifying and referring students to treatment, factors that influence their recognition of a problem and decision to refer, and the status of current research on both teachers’ role and influential factors.

Need for Teacher Training in Children’s Mental Health

Early mental health intervention is essential for maximizing students’ positive outcomes; however, a problem must first be identified for a referral to be made. Teachers play an important role in identifying students with mental health concerns. In fact, Reinke, Stormont, Herman, Puri, and Goel (2011) surveyed teachers about their role, knowledge, and training needs to support children’s mental health in schools and found 75% of respondents had worked with and/or referred students with mental health concerns in the last year. In their model describing factors that influence how children with mental health concerns access mental health care, Stiffman et al. (2010) call these teachers the gateway providers or someone who is not a mental health professional but directs or initiates access to treatment. Despite this critical role in helping children access mental health care, teachers regularly identify their need to recognize and understand children’s mental health issues as one of the top three areas in which they need training (Reinke et al., 2011).

The need for training teachers to recognize mental health concerns in the classroom may be particularly pressing for youth with internalizing behavior problems. Anxiety and depression are common internalizing disorders seen in students, affecting approximately 32% and 14% of youth, respectively (Merikangas et al., 2010). Despite their prevalence, these disorders often go undetected because of their lack of overt or observable symptoms, such as disrupting class and violating school rules. In fact, children who display externalizing problems are 20% more likely to receive mental health services than those exhibiting internalizing problems (Bradshaw et al., 2008). This phenomenon has been referred to as the “squeaky wheel,” in which children who disrupt the classroom or learning environment due to more noticeable educational and behavioral problems are more likely to receive services than students with less observable or disruptive difficulties (Bradshaw et al., 2008). This notion is supported by previous research finding adults to be better at detecting early externalizing problems in children than internalizing problems (Achenbach, McConaughy, & Howell, 1987; Glaser, Calhoun, Bradshaw, Bates, & Socherman, 2001). Additionally, researchers have demonstrated that teachers not only have difficulty identifying students with self-reported symptoms of internalizing behavior problems (Cunningham & Suldo, 2014; Neil & Smith, 2017), but are also more likely to refer and more concerned about youth demonstrating externalizing problems than those demonstrating internalizing concerns (Chang & Sue, 2003; Loades & Mastroyannopoulou, 2010).

Impact of Teacher Training in Children’s Mental Health

Several teacher training strategies are available aiming to improve teachers’ ability to recognize students’ mental health problems and manage them in the classroom. For example, Youth Mental Health First Aid has been implemented widely across the globe including content covering common mental health conditions in childhood and adolescence, prevalence rates, risk factors, warning sign and symptoms, typical development, trauma, suicide prevention and intervention, and de-escalation techniques (Jorn, Kitchener, Sawyer, Scales & Cvetkovski, 2010). Another program to improve detection of depression among adolescents includes video, small group, and case study methodology to teach in-service teachers about the sequelae of adolescent depression, importance and difficulties of early detection, signs and symptoms in a school setting, co-morbid conditions (e.g., school refusal, drug and alcohol abuse), and management strategies that could be used in a school setting (e.g., problem solving and activity scheduling; Moor et al., 2007). Researchers have examined the impact of both and other training programs and detected promising effects on participants’ knowledge, confidence, attitudes, and intentions. However, the impact on student outcomes (e.g., identification accuracy, referral rates, and/or intervention receipt) is rarely examined. When studied, some have found few, if any effects. In fact, Moor et al. (2007) found no improvement in teachers’ ability to detect students self-reported depression after training and when compared to a control group.

Gateway Provider Model

Given the continued finding that at least 25% of students with mental health concerns do not receive needed treatment (Splett et al., 2018), there is a pressing need to develop in-service training strategies that work. That is, training that increases teachers’ referral of mental health concerns to mental health professionals, who can engage children and families in appropriate treatments, is needed. Evidence supporting the Gateway Provider Model (GPM) suggests this training should consider how to influence factors beyond recognition or identification of mental health concerns. In the GPM, children’s access to mental health care is influenced by child and non-child factors, such as the teachers and other gateway providers’ (e.g., family and friends, child welfare) perception of a problem and the child’s need for treatment, access to treatment resources, and the real and perceived burden of treating mental health problems in the school and community (Stiffman et al., 2000, 2001). Children’s self-reported need for services only explained 20% of the variance in intervention receipt for models Stiffman et al. tested in education, juvenile justice, child welfare, and mental health settings. In fact, the gateway provider’s perception of need, knowledge of resources, and other organizational factors influencing their work environment more strongly predicted children’s use of mental health services than children’s self-reported need for services (Stiffman et al., 2000, 2001).

Existing Literature Supporting the Gateway Provider Model

Several studies using vignettes to describe hypothetical internalizing and externalizing behavior problems have examined teachers’ perception of children’s behavior problems and likelihood of referral. In these studies, researchers have found teachers’ perception of need varies by problem severity and type of behavior. For example, Green et al. (2018) found teachers were more concerned about severe than moderate vignettes and rated the internalizing behavior problems of a female student as more worrisome and serious than a male student with an externalizing behavior problem. Within the domain of internalizing behavior problems, Headley and Campbell (2011) found teachers were able to identify severe levels of anxiety as a problem, but had difficulty distinguishing severe levels of anxiety from moderate concerns. In contrast, Loades and Mastroyannopoulou (2010) found teachers were equally able to identify a problem and sensitive to its severity across externalizing and internalizing problems, but were more concerned with the externalizing vignettes than internalizing. However, Pearcy, Clopton, and Pope (1993) found teachers referred more children with externalizing problems than internalizing problems for services, even though they rated hypothetical scenarios of externalizing behavior problems just as severe and in need of more treatment as internalizing behavior problem vignettes. Taken together, these studies demonstrate differences in teachers’ perceptions of and beliefs about internalizing and externalizing behavior problems, their severity, and the need for intervention. The GPM would predict that these differences influence and account for differences in whether or not a teacher (as a gateway provider) refers a student for intervention. However, no prior studies have tested this hypothesis.

Current Study

Perceptions of worry, seriousness, and need for intervention are important factors in the GPM, but not commonly addressed in current efforts to train teachers to detect mental health problems. To better inform teacher training, the current study used a vignette-based survey to examine variations in teacher-reported seriousness, concern, and need for intervention between externalizing and internalizing behavior problems. Additionally, the influence of teacher-reported seriousness and concern on their endorsement of the child’s need for intervention was examined. More specifically, the current study answered the following research questions:

  1. (1)

    Do teachers accurately identify when children are exhibiting externalizing and internalizing behavior problems and are they sensitive to variations in severity of the problem displayed?

  2. (2)

    Do teachers’ ratings of seriousness of the problem, concern for children, and endorsement of children’s need for intervention differ by problem type?

  3. (3)

    Do teachers’ rating of seriousness and concern for children predict their endorsement of children’s need for intervention when controlling for child gender?

We hypothesized teachers would be able to accurately identify the existence of a behavior problem and be sensitive to the problem’s severity regardless of the problem type described given prior research (Loades & Mastroyannopoulou, 2010). However, we also hypothesized teachers would rate the seriousness of the problem, concern for the child, and endorsement of the child’s need for intervention differently by problem type. Given prior research detecting similar differences and known differences in mental health service use by problem type, we expected teachers to rate children displaying externalizing behavior problems as having more serious problems, being more concerned for, and more likely to endorse the child’s need for intervention than children displaying internalizing behavior problems (Bradshaw et al., 2008; Green et al., 2018; Headley & Campbell, 2011; Loades & Mastroyannopoulou, 2010). Finally, based on factors included in the GPM (Stiffman et al., 2004), we hypothesized that teachers’ ratings of problem seriousness and concern for the child would predict their endorsement of the child’s need for intervention even when controlling for gender. In the current study, we controlled for gender because prior research has either confounded gender with problem type (Green et al., 2018) or found gender to be a significant predictor of teachers’ ratings of perceived need for treatment (Green, Clopton, & Pope, 1996; Loades & Mastroyannopoulou, 2010).

Method

Participants and Setting

The participants in this study were teachers from four public schools participating in a federally funded, randomized controlled trial (Award No. 2015-CK-BX-0018). Study schools were in a large southeastern school district in the USA. A total of 153 teachers participated, including three in pre-K and the remaining 150 teaching in grades K to 5. The majority of participants in this sample were European American (75%) and female (90%). The participants’ years of teaching experience ranged from 1 to 5 years (22.2%) to 20 or more years (26.1%). Many of the participants had bachelor’s (68.6%) or Masters’ degrees (23.5). Full demographic information for study participants is reported in Table 1.

Table 1 Demographic characteristics of participating teachers
Full size table

Procedures

The study was conducted with schools randomized to the intervention condition of the Project About School Safety, a four-year, federally funded, randomized controlled trial of the Interconnected Systems Framework (ISF). The ISF is a multi-tiered intervention strategy connecting school mental health with school-wide Positive Behavioral Interventions and Supports (PBIS) through data-based decision making, interprofessional teaming, implementation support for evidence-based practices, and ongoing quality improvement to assure responsiveness to school and student needs (see Barrett, Eber, & Weist, 2013). Procedures for the current study were approved by the investigators Institutional Review Board and the school district and implemented during the first three months of ISF implementation (Fall 2016). Because all survey data were collected anonymously, waiver of written consent, also called passive consent, procedures were approved by the IRB and used to conduct the study.

A quantitative, cross-sectional design was utilized to assess teachers’ perceptions and their abilities to recognize developmentally common childhood mental health problems. A paper questionnaire was created based on existing measures, composed of a closed and open-ended questions regarding teachers’ experience, knowledge, and training related to children’s mental health needs followed by a series of vignettes assessing teachers’ recognition of children’s mental health problems and their help-seeking behaviors. Two versions of the vignette measure were used, and teachers were randomly assigned to receive one of the two survey versions via the survey proctor. Teachers completed the paper-based survey onsite in large group settings in approximately 10 min. As an incentive for participating, teachers who completed the survey were entered into a drawing to win one of several gift baskets including classroom supplies worth approximately $75 per basket. The data were manually entered by trained undergraduate research assistants with 97% reliability. Two advanced graduate research assistants resolved discrepancies by reviewing the survey and reaching consensus.

Measures

Teacher Survey

The Teacher Mental Health Literacy and Practices Survey was developed from two existing measures to assess teachers’ knowledge, past experiences, training, recognition, concern, and referral strategies for children’s mental health problems. The survey consisted of 61 items with four different response formats (i.e., dichotomous, Likert scale, multiple choice, and fill in the blank). It included nine demographic questions, 16 items from the Mental Health Needs and Practices in Schools Survey (Reinke et al., 2011) querying teachers’ training, knowledge, and experience with children’s mental health problems in their classroom, and 36 vignette-based items from the Teachers’ Mental Health Literacy Questionnaire (Loades & Mastroyannopoulou, 2010) measuring their ability to identify common mental health concerns and intervention need.

For the purposes of the current study, only items from the Teachers’ Mental Health Literacy Questionnaire were used in data analyses to answer research questions. As reported in Loades & Mastroyannopoulous (2010), the Teachers’ Mental Health Literacy Questionnaire was used in prior research with teachers following pilot testing and development with advanced clinical psychology doctoral students assumed to have an advanced understanding and knowledge of externalizing and internalizing behavior problems. In most prior research, the Teachers’ Mental Health Literacy Questionnaire demonstrated adequate reliability and validity (Jacobs & Loades, 2016; Loades & Mastroyannopoulou, 2010). However, 25 to 42% of education and health professionals identified a problem in two vignettes intended to describe the absence of behavior problems in both pilot testing and surveys with teachers (Jacobs & Loades, 2016; Loades & Mastroyannopoulou, 2010). Therefore, these two vignettes were not used in data analyses for the current study due to poor discriminant validity in prior work.

Items drawn from the Teachers’ Mental Health Literacy Questionnaire included a series of vignettes that described common externalizing (i.e., Oppositional Defiant Disorder) and internalizing (i.e., Separation Anxiety Disorder) behavior problems for children in elementary school (see examples in Table 2). Following each vignette were six questions querying (1) respondents’ recognition of the problem (dichotomous yes/no), (2) rating of problem severity (categorical, three-point Likert scale), endorsement of the child’s need for help from both (3) school-based mental health professionals (dichotomous yes/no), (4) community-based mental health professionals (dichotomous yes/no), (5) level of concern for the child described in the corresponding vignette (categorical, five-point Likert scale), and (6) prompt to name the problem (fill in the blank). Two versions of the measure were administered in which the gender of the child in each vignette was counterbalanced, but all other contents remained the same. Each version of the vignette survey included six vignettes with three describing severe, moderate, and problem-free levels of an externalizing behavior problem and three describing severe, moderate, and problem-free levels of an internalizing behavior problem. In version 1, the children in all three vignettes describing externalizing behaviors were male and the children in all three vignettes describing internalizing behaviors were female. In version 2, the children in all three vignettes describing externalizing behaviors were female, while the children in all three vignettes describing internalizing behaviors were male. Seventy-seven respondents completed version 1 and 76 completed version 2. Only responses to the vignettes describing severe and moderate externalizing and internalizing behavior problems were retained for further analyses.

Table 2 Vignette examples
Full size table

Data Analyses

We carried out all analyses using R (R Core Team, 2017). For the first research question, we performed McNemar’s tests to determine whether there were significant differences in the proportion of correct identification for each level of problem behavior and a Wilcoxon signed rank-sums test (Wilcoxon test) to determine whether they correctly identified the clinical vignettes as more serious than the subclinical vignettes. The McNemar’s test uses the paired identification responses for the external and internal vignettes within each problem level and results in a Chi-square statistic with one degree of freedom. We chose to perform the test with a continuity correction due to the chance for some of the paired response options to have few observations. The Wilcoxon test was chosen for this and subsequent research questions because the test does not assume a normal distribution on the independent variables and allows for ordinal variables. Since the response variables were either binary (correct or incorrect recognition of a problem, yes or no endorsement of need for help from school and/or community-based mental health professionals) or polytomous (scale of 0–2 for seriousness, scale of 0–4 for concern), the assumptions of interval measurement and normality required for the paired-samples t test are not met. Thus, the Wilcoxon test was a natural choice for our data and research questions.

To answer the second research question, we performed a series of Wilcoxon tests. Teachers’ answers to the survey questions about each vignette were matched and analyzed to determine whether there were significant differences in their ability to identify problem behaviors by problem type, level of concern by problem type, and likelihood of referring by problem type.

As teachers were asked whether the children in each vignette should be referred to (1) mental health professionals employed by the school and to (2) mental health professionals in the community, these were analyzed again for the third research question.

We estimated a measure of the effect size for any significant difference using

$$r = \left| {\frac{Z}{\sqrt N }} \right|,$$
(1)

where Z is the approximate z score for the test statistic from the Wilcoxon test and N is the total number of matched observed scores. This value indicates the degree to which the median values of the two questions being compared differ. We used the conventional values of a small effect being greater than 0.1, a medium effect being greater than 0.3, and a large effect being greater than 0.5 (Gray & Kinnear, 2012).

For the Wilcoxon tests, we hypothesized that teachers would correctly rate the severe vignettes as more serious than the moderate vignettes, express more concern for children displaying externalizing behavior problems, and be more likely to refer children displaying externalizing behaviors, so these tests were one-sided. All significance levels for these hypotheses were adjusted using the Bonferroni adjustment to control the familywise error rate within each research question.

It was expected that the differences between some responses would be so statistically significant that we would run into problems estimating the true magnitude of the difference. Because the effect size is calculated from the Z statistic, which is in turn calculated from the p value of the Wilcoxon test, the estimates for these tests are bounded by the accuracy in the p values. The minimum p value reported by R (\(2 \times 10^{ - 16}\)) could reasonably be expected given the results in previous research. Additionally, as the sample size increases, the effect size statistic decreases while holding Z constant; since Z had an upper bound on its maximum value, and the sample size in this study was larger than in previous studies (e.g., Jacobs & Loades, 2016), the effect sizes would have a largest possible value of approximately r = 0.66 in these data. Thus, the reported Z statistics and effect sizes should be interpreted as the lower bound estimates of the effect and may be approximately the same in many of the reported comparisons.

For the third research question, a series of logistic regressions were used to predict teacher endorsement of children’s need for intervention, using the teacher ratings of seriousness and concern for the children and controlling for the gender of the child in the vignette each teacher saw.

Finally, as there was intermittent missingness in the responses to the survey, we used a pairwise deletion strategy to maximize the number of observations in each analysis. In this way, when there were no missing values on the variables used for a particular analysis, the full sample was used, while if some variables have missing values only the analyses they are used in had a reduced sample. When data are missing, we report the number of observations used.

Results

Descriptive statistics, including proportions, standard errors, means, and standard deviations, of all study variables are reported in Table 3. For the variables used in the study, there were between 110 (71.9%) and 153 (100%) responses. Although each teacher responded to the questions regarding the presence of a problem, some did not answer the follow-up questions.

Table 3 Means/proportions and standard error/deviations of study variables
Full size table

Teacher Accuracy of Problem and Severity

Problem Identification Accuracy

The McNemar’s tests compared the following four accuracy conditions at each level of problem severity (i.e., severe, moderate, and problem free): correctly identifying a problem in both the externalizing and internalizing vignette, correctly identifying a problem in one of the vignettes but not the other (two possible outcomes), and incorrectly stating there was no problem in both vignettes.

The test for the severe vignettes determined there was not a significant difference in accuracy of teacher identification of externalizing (100%, n = 153) and internalizing (98%, n = 150) problem behaviors (\(\chi_{1}^{2} = 1.33, p = 0.248, N = 153\)), indicating that teachers were able to accurately identify severe problems regardless of whether they were externalizing or internalizing.

The results for the moderate vignettes indicated a significant difference between problem types in teacher accuracy (\(\chi_{1}^{2} = 18.225, p < .001, N = 153\)). The proportion of correct identification was higher for the externalizing vignette (91%, n = 139) than for the internalizing (73%, n = 111), indicating that teachers were less accurate in correctly identifying a moderate internalizing behavior problem than externalizing.

Problem Severity

The Wilcoxon test showed that, within each problem type, teachers rated the seriousness of the severe vignettes significantly higher than the seriousness of the moderate vignettes (externalizing behavior problems Z = − 7.94, p < .001, r = 0.65; internalizing behavior problems Z = − 7.94, p < .001,\(r = 0.65\)). Thus, teachers were able to distinguish between severe and moderate problems across both externalizing and internalizing vignettes.

Teacher Ratings of Problem Seriousness, Concern, and Need for Intervention

Seriousness

Between problem types, teachers rated the severe externalizing behavior vignette as being more serious than the severe internalizing, Z = − 7.75, p < .001, r = 0.64. They also rated the moderate externalizing behavior vignette as more serious than the moderate internalizing vignette, Z = − 7.74, p < .001, r = 0.63. Regardless of problem severity, teachers rated externalizing behavior problems as more serious than internalizing.

Concern

Teachers rated their concern for children in the severe externalizing behavior vignette (M = 4.81, SD = 0.45) as higher than their concern for children in the severe internalizing behavior vignette (M = 3.99, SD = 0.86), Z = − 7.69, p < .001, r = 0.64. They also rated their concern for the children presenting moderate externalizing behavior problems (M = 4.04, SD = 0.86) as higher than their concern for the children with moderate internalizing problem behaviors (M = 3.25, SD = 1.00), Z = − 7.41, p < .001, r = 0.61. Again, regardless of problem severity, teachers rated their concern for the child’s well-being higher when the child displayed externalizing behavior problems than when the child in the vignette displayed internalizing behavior problems.

Need for Intervention: Professional Help Within the School

Out of 148 responses, 142 teachers in the sample endorsed the need for intervention from a school-based mental health professional for children in the severe externalizing behavior vignette (96%, SE = 0.02), while 132 of 149 teachers endorsed the same for children in the severe internalizing behavior vignette (89%, SE = 0.03). The Wilcoxon test indicated a significant difference between problem types, Z = − 2.40, p = .008, with a small effect size of r = 0.20. Responding to the moderate externalizing behavior vignette, 127 of 140 teachers endorsed the need for intervention from a school-employed mental health professional (91%, SE = 0.02) compared to 96 out of 113 teachers responding to the moderate internalizing behavior vignette (85%, SE = 0.03). The associated Wilcoxon test showed a significant difference between problem types, Z = − 2.21, p = .013, with a small effect size of r = 0.21.

Need for Intervention: Professional Help from the Community

A total of 146 out of 149 teachers endorsed the need for intervention from a mental health professional in the community for children displaying severe externalizing behaviors (98%, SE = 0.01), while 131 out of 148 endorsed the same for children displaying severe internalizing behaviors (89%, SE = 0.03). The Wilcoxon test indicated a significant difference between problem types (Z = − 2.92, p = .002) with a small effect size (r = 0.24). For the moderate problem behaviors, 121 out of 140 teachers endorsed the need for help from a community mental health professional for children displaying externalizing behaviors (86%, SE = 0.03), while 73 out of 112 endorsed the same for children displaying internalizing behavior problems (65%, SE = 0.05). The associated Wilcoxon test showed a significant difference between problem types, (Z = − 4.62, p < .001) and a medium effect size of r = 0.45.

Factors Influencing Need for Intervention

Severe Externalizing and Internalizing Problem

All logistic regression results are reported in Table 4. For the severe problem vignettes, the teacher level of concern was a significant predictor in the probability of endorsing the need for referral to school mental health professionals. The magnitude of this effect was similar for both the externalizing (β = 1.518, p = .038) and the internalizing (β = 1.380, p = .002) vignettes and indicates that for a one-point increase in the level of concern a teacher reported for a student, the odds of endorsing the need to refer the student to a school mental health professional are 4.561 times larger for the externalizing problem and 3.973 times larger for the internalizing problem, after controlling for child gender. In both cases, the teachers’ ratings of the seriousness of the problem were not associated with an increase in the probability of referral to school mental health professionals.

Table 4 Teacher endorsement of need for intervention predicted by concern, seriousness, and child gender
Full size table

Neither the teachers’ concern nor their rating of the seriousness of the problem was associated with an increased probability of endorsing the need to refer the child to a community mental health professional in the severe externalizing problem vignette. However, for the severe internalizing problem vignette, concern was associated with an increased probability of endorsing need for referral (β = 1.188, p = .007) after controlling for child gender. The ratings of the seriousness of the internalizing problem were not significant. Thus, for the internalizing problem vignette, for each one-point increase in concern, there is a 3.282 times increase in the odds of a teacher endorsing the need to refer the child to a community mental health professional after controlling for child gender.

Moderate Externalizing and Internalizing Problem

Neither the level of concern nor the rating of the seriousness of the problem was associated with an increased probability in referring the child to school professionals in the moderate externalizing problem vignette after controlling for child gender. However, for the moderate internalizing problem vignette, teacher concern was a significant predictor of the probability of teachers endorsing the need for referral to a school mental health professional (β = 1.242, p = .004) after controlling for child gender. The odds of referral increased by 3.463 times for each one-point increase in teacher concern. Ratings of the seriousness of the internalizing problem were not associated with an increase in endorsements of referral to a school mental health professional.

Teacher concern for the moderate externalizing behavior problem vignette was related to an increase in the probability of endorsing the need to refer to community mental health professional (β = 1.354, p = .006) after controlling for child gender, but ratings of the seriousness of the problem were not. For every one-point increase in concern, the odds of community referral increase by 3.874 times. Finally, the ratings of the seriousness of the internalizing problem is related to an increase in the probability of referring to community professionals (β = 2.043, p < .001) after controlling for gender. However, the teachers’ concern was not associated with an increase in community referral. For every one-point increase in teacher rating of the seriousness of the moderate internalizing problem vignette, there was a 7.710 times increase in the odds of endorsing the need for referral to a community mental health professional.

Discussion

The current study evaluated teacher ability to accurately identify externalizing and internalizing problems in elementary age children using vignettes that counterbalanced child gender and presented two levels of problem severity. In the GPM, teachers drive whether or not a student is referred for services and are influenced by factors examined in the current study, including the perceived seriousness of the problem, concern for the student, and/or knowledge of resources they believe might help the student (Stiffman et al., 2004). Results suggest differences between teachers’ recognition of, perceived seriousness of, concern for, and endorsement of the need for help between students with externalizing and internalizing problems.

As with prior research (Loades & Mastroyannopoulou, 2010), teachers were able to accurately identify both externalizing and internalizing problems when the problems were severe. In addition, teachers were able to accurately distinguish between severe and moderate problems. However, teachers in this sample had more difficulty accurately identifying moderate or subclinical internalizing behavior problems, which raises the possibility of educators failing to identify students in need of help in time to prevent problem worsening and disordered behavior. This is concerning because early risk factors associated with mental health problems such as externalizing and internalizing problems lead to a host of negative later life outcomes that become entrenched and more difficult to intervene upon the longer they persist (Soni, 2009). Early detection and prevention are important (Reinke et al., 2018). Helping support teachers in being able to identify early malleable risk indicators is important for ensuring students with moderate internalizing behavior problems have access to prevention and early intervention resources. These results suggest teaching recognition of subclinical or moderate internalizing behavior problems to teachers is a critical area of future research and practice.

Beyond problem recognition, the GPM (Stiffman et al., 2004) suggests perceived seriousness of the problem, level of concern for the student, and knowledge of available resources to help the student would make it more or less likely for a teacher to make a referral. As hypothesized, when controlling for student gender, teachers found externalizing problems to be more serious and more concerning than internalizing problems. This may be due to the fact that behaviors associated with externalizing problems (i.e., aggression, defiance, noncompliance) are deemed more disruptive to the classroom environment. Many teachers report feeling unprepared for handling challenging externalizing behavior in the classroom (Reinke et al., 2011). In fact, nearly half of teachers leave the profession early and indicate disruptive student behavior as the primary reason (Ingersoll, 2002). Relatedly, teachers endorsed the need for referral to school and community mental health professionals at a significantly higher rate for externalizing behavior problems than internalizing.

These findings are in line with prior research which found that students with disruptive problems were more likely to be identified and provided with school and community services than students with internalizing problems (Bradshaw et al., 2008; Splett et al., 2018). Bradshaw et al. referred to this as the “squeaky wheel” phenomenon, meaning that those students who present teachers with more difficulties in the classroom will be more likely to be referred to and targeted for services. Although internalizing behavior problems may not be a “squeaky wheel” in the classroom, they do cause substantial proximal and distal disruption in students’ lives. Future research and practice should consider ways in which teaching educators about these personal disruptions may elevate their perceived seriousness and concern for students with internalizing behavior problems.

Additionally, teachers were less likely to refer moderate levels of internalizing behavior problems to community-based help than school-based help. Prior research has also found students with externalizing concerns accessed community-based psychosocial care at higher rates than students with internalizing concerns (Splett et al., 2018). Low rates of referrals to and access of community-based help may highlight an area of available resources that are not actively used by schools to address the mental health concerns of their students. It may be that teachers and school-based providers are unaware of these available resources or how they might help students with internalizing behavior problems. Types of community-based resources available and how to access them may be an area of training that could benefit teachers overall mental health knowledge and increase referral rates. This may be particularly important for students with internalizing concerns given prior research has shown knowledge of such resources predicts referrals (Green et al., 2013; Stiffman et al., 2000), and identification and encouragement to seek mental health services predicts service use, but is more likely for students with externalizing problems than internalizing problems (Alegría et al., 2012).

Lastly, this study went beyond prior research to investigate how teachers’ perceptions of how serious a problem was or how concerned teachers felt for a student’s well-being impacted their perception of the student’s need for intervention. Based on the Gateway Provider Model, it was hypothesized that higher levels of perceived seriousness or concern would be associated with the likelihood that a teacher would consider the student in need of intervention either by the school or in the community. Findings indicated that when teachers had high levels of concern for the child’s well-being they were more likely to endorse the need to refer the student for school and/or community-based mental health services, even when controlling for the child’s gender. This was true for both externalizing and internalizing behavior problems at moderate and severe levels. As previously reviewed, however, teachers were less concerned for the child’s well-being when the child displayed internalizing behaviors rather than externalizing concerns. Given this difference and the influence of teachers’ concern on their likelihood of referring children for mental health, future research and training efforts should consider focusing more attention on strategies that impact teachers’ concern for students struggling with internalizing behavior concerns. Achieving this may be particularly impactful for students with internalizing problems given they are most likely to not receive needed services (Bradshaw et al., 2008; Splett et al., 2018).

Limitations

Although this study gleaned some important findings, there were also limitations. This study is limited by the fact that the data set utilized self-report from a sample of elementary teachers in one school district and one state. These teachers have likely had very similar pre-service and in-service training experiences, and prior research has shown differences in referral rates by grade levels with elementary school teachers endorsing intervention referrals at lower rates than high school teachers (Green et al., 2018). Therefore, results may not generalize to all educators across states and grade levels. Further, the use of vignette methodology makes it difficult to evaluate how teachers might respond to actual cases or real students. The vignettes in this study also explored some of the most common childhood internalizing (SAD) and externalizing problems (ODD), but these may not be indicative of all issues experienced by children and seen by teachers in the classroom. Additionally, the use of vignettes to approximate referral decisions may not accurately capture what teachers would do in real life when they likely know their students better and have a personal connection, which would theoretically increase referral rates, and/or are less aware or concerned about a problem that develops across time rather than being written in a succinct paragraph, which might decrease referral rates. Relatedly, the study only measured referral to mental health professionals as an intervention option when a wider range of intervention strategies are likely available to most teachers and study participants (e.g., provide positive reinforcement, reduce homework, provide additional time, request assessment). Due to psychometric concerns, we did not conduct analyses with the problem-free vignettes. Given limited resources in schools and consistent findings that about 20 to 30% of educators and mental health professionals identify problems in problem or symptom-free vignettes (Day, 2002; Jacob & Loades, 2016), future research examining teachers’ interpretation of normal or typical student behavior is needed. Future research in this area could also include data from universal screening and direct observation of students to confirm elevated risk and then be triangulated with teacher self-report of seriousness and concern to examine the impact on their endorsement of a wider range of intervention strategies, both within and outside of the classroom.

Conclusion

The study findings are timely and needed. Student mental health problems are an area being given a great deal of attention. Many teachers feel underequipped to support students experiencing mental health problems, but pressure for schools to be more responsive to students’ mental health needs continues to mount. Finding ways to support teachers in identifying and moving toward action to bring early concerns to the attention of school-based mental health providers is needed. By identifying early malleable risk indicators of more severe externalizing and internalizing behaviors, schools and communities can work to prevent more significant mental health issues among our youth. Universal screening and teacher training to identify mental health symptoms are viable strategies, but results here suggest finding methods that impact teachers’ concern for the students’ well-being, understanding of available resources, and the need to act early may be most impactful in getting prevention and intervention services to those students in need. These strategies might be most critical for youth with internalizing behavior problems given teachers reported them as less concerning and students with such problems access care at significantly lower rates. Across problem type, working in schools where children are present for the majority of the day and week, and with teachers as gateway providers directing access to care is paramount to more effectively and efficiently meeting children’s mental health prevention and intervention needs.