Introduction

Though not part of their formal mandate, schools in the United States have increasingly come to play an important role in addressing children’s mental health (McLaughlin, Leone, Meisel, & Henderson, 1997). Seventy percent of psychosocial services provided to children take place in school settings (Rones & Hoagwood, 2000). Schools also function as critical sites for the early identification of mental health problems (New Freedom Commission on Mental Health, 2003), in part because teachers are often the first to notice the signs in students, particularly when they manifest in disruptive, aggressive, or antisocial behaviors (Roeser & Midgley, 1997). Further, school settings are well-suited for the implementation of universal interventions to promote positive mental health as well as targeted interventions for children at greater risk for specific problems, such as substance use or depression (Greenberg, 2003; Noam & Hermann, 2002).

As attention on schools’ role in providing mental health services for children has increased, the federal government has called for the accelerated development of school mental health programming (Substance Abuse and Mental Health Services Administration, 2007; U. S. Department of Health and Human Services, 1999). Some comprehensive or “expanded” school mental health programs provide a continuum of mental health services in schools, including treatment, early intervention and prevention programs, and consultation and staff training (e.g., Atkins et al., 2006; Flaherty & Weist, 1999; Watts & Buckner, 2007).

For the most part, evaluation of these types of interventions has tended to focus on individual-level change (Hoagwood et al., 2007). Outcomes of interest have included student academic performance, behavioral functioning, and mental health. A few evaluations have also attempted to assess the impact on teachers, such as increased knowledge of mental health issues or skills for behavior management. At the setting level, some intervention research has looked at the school climate; mental health interventions often demonstrate change in the quality of interpersonal and intergroup relationships, the perceived safety (both emotional and physical), or student behavior. It has been more challenging to measure the impact of a mental health intervention at the school-level of analysis. This may be due, in part, to the relative lack of assessment tools that capture setting-level characteristics (Tseng & Seidman, 2007), especially around mental health.

In this paper, we describe a new measure that captures, at the school-level of analysis, a school’s ability to address the mental health needs of its students, which we call its “mental health capacity.” First, we define this term, drawing from literature in community psychology, prevention science, and school mental health. Next, we describe the development of the “School Mental Health Capacity Instrument” (SMHCI), which measures this construct, focusing on systems-level variables such as the presence of structures, protocols, and policies that address student mental health. We then present initial findings regarding the instrument’s psychometric properties, including internal consistency, test–retest reliability, and validity. Lastly, we outline the instrument’s potential uses for research and practice in school mental health.

School Mental Health Capacity

Capacity, in its broadest sense, refers to aspects of an organization or community that enable it to function efficiently as a system as it works to fulfill its mission (USAID Center for Development Information and Evaluation, 2000). An organization’s mental health capacity is more limited in scope, pertaining to its ability to address mental health in the service of its goals and objectives. A school’s mental health capacity, then, is its ability to address children’s mental health issues in the context of its primary function as an educational organization. We believe schools with greater mental health capacity can also be described as being proactive in their approach to mental health.

Our thinking about schools as being “proactive” is heavily influenced by the idea of “proactive coping” strategies at the individual level (Aspinwall, Sechrist, & Jones, 2005; Aspinwall & Taylor, 1997). People who engage in acts of proactive coping recognize that stressors may occur, adopt measures to identify them sooner rather than later, and take steps ahead of time to prevent their occurrence or mitigate their impact (Aspinwall & Taylor, 1997). A more proactive approach to dealing with stress involves developing systems of support and resources that may counteract the possible negative impact of future stress (Hobfoll, 1989). Proactive coping can also be thought of as “challenging the status quo rather than passively adapting to present conditions” (Crant, 2000, p. 436); it requires consideration of what may be in the future and action to improve the current circumstances.

We argue that this concept can also be viewed at an organizational level of analysis. There is little research that explicitly discusses “proactive schools,” with the exception of Perry (1999), who frames proactive in the context of school safety and disciplinary policies. Related to mental health, we argue that more proactive schools recognize that mental health problems may occur, put systems into place to recognize them sooner rather than later, and take steps to prevent them from occurring or to reduce their impact on the school when they do occur. These efforts are similar to the levels of prevention (indicated, selective, and universal) outlined in the Institute of Medicine’s Report (National Research Council and Institute of Medicine of the National Academies, 2009). These different levels of prevention, and Caplan’s (1964) tertiary, secondary, primary prevention rubric, reflect the notion of addressing problems in a somewhat reactive manner on one end to increasingly proactive approaches on the other end. We outline our thinking in each of these areas below.

At the universal level, are activities and structures that schools implement to address mental health for all students (National Research Council and Institute of Medicine of the National Academies, 2009). Schools that are more proactive may implement interventions that promote positive development (Brown, Roderick, Lantieri, & Aber, 2004; Kusche & Greenberg, 1994), support positive social behavior (Flannery, Sugai, & Anderson, 2009), or build a sense of community within the school (Battistich, Schaps, & Wilson, 2004). Efforts to promote wellness are integrated throughout the fabric of the school by creating regular opportunities for staff, students, and families to learn and talk about mental health-related issues.

Proactive schools also work to prevent some problems sooner rather than later by implementing selective interventions (National Research Council and Institute of Medicine of the National Academies, 2009). Staff in more proactive schools recognize that mental health issues may affect children’s learning as well as their ability to function in the classroom environment. They may intervene early with students at risk for developing mental health issues (Greenberg, Domitrovich, & Bumbarger, 2001). In addition, teachers can identify the early signs of mental health problems and know what to do when they have a concern about a student. There is a system for gathering referrals from teachers as well as a protocol for connecting students with appropriate services, communicating information back to teachers, and monitoring the situation on an ongoing basis.

Proactive schools recognize the need for indicated interventions for those students in need (National Research Council and Institute of Medicine of the National Academies, 2009), but also for the schools to have plans in place to respond quickly and efficiently when these types of crises arise. Even with efforts on prevention and early intervention, all schools will inevitably experience mental health emergencies, such as students who express suicidal thoughts or who become out of control in the school environment. A professional is available to provide consultation in these situations, either in the school or through a community partner. Further, more proactive schools continue to follow up after crises, monitoring the student and family as well as addressing any new problems that may arise. Schools that are prepared to handle such situations may be able to reduce their negative impact, much the same way that individuals’ proactive coping skills are associated with more positive outcomes in the face of stressors (Mallett & Swim, 2005).

At the other end of the continuum from proactive are schools that we would best describe as “reactive” in their approach to mental health. In these schools, there is little integration of mental health into the academic environment; prevention activities or social–emotional curricula are implemented erratically or not at all. It is not uncommon for reactive schools to have punitive measures in place to respond to problems, but such measures have been shown not to be effective in the long-term (Perry, 1999). In these schools, staff receives minimal training about how to identify students who may be demonstrating early warning signs of mental health problems in the classroom. When students in need of additional support are identified, there is no systematic way to refer them for services, share information about them, or monitor their progress. When emergency situations occur, there is no shared understanding of the procedures to follow in that moment; nor is there much in the way of debriefing or follow up afterward. Schools on this end of the continuum tend to spend much of their time reacting to problems, as opposed to building an environment that cultivates positive mental health and well-being.

A school with greater mental health capacity—one that utilizes a more proactive stance toward mental health—may ultimately function more effectively and efficiently as both an organizational system and a learning environment. Emergencies cause less disruption when schools have crisis intervention plans that clearly delineate tasks and responsibilities (Johnson, 2000). Schools that have structures in place that enable earlier recognition of problems are able to intervene sooner rather than later in the course of a student’s mental health problem, which decreases the likelihood of problems developing to a crisis level (Levitt, Saka, Romanelli, & Hoagwood, 2007). Mental health-related prevention and promotion activities may facilitate students’ academic achievement (Zins, Bloodworth, Weissberg, & Walberg, 2004).

In sum, we maintain that the key component underlying a school’s mental health capacity is the extent to which it is proactive in its approach to mental health. The School Mental Health Capacity Instrument is intended to provide a way to measure this construct. The instrument can also help identify areas where schools can focus efforts to build mental health capacity over time.

Related Measures

There are a growing number of instruments focused on the school as a setting. Some measures of schools as settings focus on the effectiveness of the learning environment, such as the instructional practices or the pedagogical actions present in the building (Hoy, 1990; Stockard & Mayberry, 1992). Other assessments examine the school “climate,” generally referring to the quality of the relationships in the building, the fairness and clarity of school rules and the perceived level of safety (Brand, Felner, Shim, Seitsinger, & Dumas, 2003; Gottfredson, 1984; School Development Project, 2001). The School-wide Evaluation Tool (Horner et al., 2004) is another school-level assessment designed to measure the implementation of school-wide positive behavior support procedures. The items focus on the specific practices and features of such a system, and the extent to which they are utilized across the school.

A small, but growing, inventory of instruments also exists to describe mental health services in schools. The “School Mental Health Quality Assessment Questionnaire” (Weist, Stephan, Lever, Moore, & Lewis, 2006) helps assess areas of strength and weakness in school-based mental health services. It was designed to align with principles for best practice in the field (Weist et al., 2005). Intended to be used by the providers working within a school, its primary purpose is for quality improvement activities.

Several instruments also have been designed as self-study tools. The “Mental Health Planning and Evaluation Template (MHPET)” was originally developed by the National Assembly for School-Based Health Care to measure the availability, quality, and integration of mental health services within school health centers. The tool is meant to provide a way to systematically assess the quality of school-based mental health services. Similar to the SMH-QAQ, the MHPET also identifies areas of strength and those in need of improvement.

The “School Health Index,” developed by the Centers for Disease Control and Prevention, is a self-assessment and planning guide that schools can use to improve their health and safety policies and programs (Staten et al., 2005). The SHI has been applied to a nationally representative sample of schools to describe how schools across the country are achieving the school health recommendations described in the tool (Brener et al., 2006). A modified instrument that includes efforts specifically focused on mental health has been used in three states.

The Center for Mental Health Services in Schools also developed a toolkit for schools to examine the services that are in place, including classroom learning supports, crisis response and prevention, support for transitions, family involvement, community involvement, and student assistance (Adelman & Taylor, 2006). The tool is meant to provide a picture of the extent to which resources are needed, available, and effective as well as to make recommendations and inform decisions about resource allocation.

While these types of instruments are intended to provide schools with a snapshot of their existing resources as well as a guide for a planning process, their emphasis is not quantitative. Measures that adapt a quantitative approach, such as the instrument we describe in this paper, facilitate an examination of the psychometric properties of an instrument, allow comparisons across schools in terms of relative degree of mental health capacity, provide a tool that can be used in examining the relationship of school mental health capacity to other variables, and enable school mental health capacity to be examined as an outcome variable in the evaluation of school-level interventions.

In sum, research and practice could benefit from the development of additional assessment tools that help to describe school-level characteristics, including those pertaining to how schools function as systems to address the mental health of students. We developed the School Mental Health Capacity Instrument (SMHCI) to help address this need. The focus of the SMHCI is on a higher order construct, namely the overall “capacity” of a school to address students’ mental health needs, may help in efforts to assess and describe schools in a more ecological and holistic manner. Focus on the school as the unit of analysis can also facilitate interventions that target the school as a whole for change thereby broadening the range of initiatives beyond those that address change at the individual or classroom-levels. In the pages that follow, we describe our method for developing the instrument and report its psychometric properties. We then discuss potential uses of the instrument in research as well as for promoting mental health capacity through consultation to a school.

Method

Instrument Development

The SMHCI was developed and tested iteratively, in two phases, over a two academic year period. In the first phase of instrument development (2006–2007 academic year), we wrote a set of 38 items to pilot. [As will be described later on, a new version of the instrument containing 27 items was put back into the field in phase two (2007–2008).] The content and language of the items were informed largely by interviews and focus groups with school-based mental health clinicians and student support staff as well as a survey of the literature on school-based mental health. The items were designed to assess the presence of specific structures within a school related to a school’s ability to address student mental health issues. The items were written such that they would have generalizability across most schools focusing more on whether or not a school could address a mental health need among students than the precise manner in which it was accomplished. Also, we emphasized the assessment of characteristics of a school that were sustainable over time, as opposed to the amount of services available only through (or because of) a certain individual or group of people. The measure was specifically designed to measure this school-level construct. Response options for these core items range from 0 = no, we don’t have or do this; 1 = yes, we have or do this a little bit; 2 = yes, we have or do this to some extent; and 3 = our school has this in place or does this to a great extent.

The organization of the instrument and its three a priori conceived subscales was intended, in part, to reflect the levels of the public health pyramid: intervention, early recognition, and prevention (Caplan, 1964; Mrazek & Haggerty, 1994) or tertiary, selective, and universal intervention (National Research Council and Institute of Medicine of the National Academies, 2009). We conceptualized each domain as collectively contributing to a school’s overall mental health capacity, while also functioning somewhat independently from each other. Items within a subscale were intended to reflect indicators of a proactive approach to addressing students’ mental health specifically related to that domain. For example, items in the intervention subscale were meant to assess the spectrum of policies or protocols a school may have in place to recognize, respond to, and follow up when urgent mental health problems arise. Respondents rate the extent to which their school has each specific policy, protocol, or system related to mental health capacity.

The instrument also contains supplementary questions that are not calculated into the total “mental health capacity” score; hence, we do not report results on these elements of the instrument. One set of questions, which is primarily intended to orient respondents to each section of the instrument, asks about the concrete resources and services available internally to the school as well as those provided by community partners. This first set of ancillary questions gather information about the specific ways that a school addresses student mental health in a mostly open-ended manner, whereas the 27 core items have fixed response options that facilitate quantification of responses. These core items get more at whether or not a school has programs and policies in place or can address a mental health need, whereas the supplementary items get more at the details of how such services or programs are delivered.

A second set of supplementary questions asks respondents to rate their satisfaction with the amount of mental health resources available, their familiarity with what is present in the school, and their level of confidence in their own ability to address mental health. While not directly relevant to the assessment of school mental health capacity, these questions provide additional information that could be useful in working with a school as a consultant.

Research Context

This research was conducted as part of ongoing research and evaluation activities for the Children’s Hospital Neighborhood Partnerships (CHNP), the community mental health outreach arm in the Department of Psychiatry at Children’s Hospital Boston. CHNP provides treatment services, prevention and early intervention activities, and training and consultation to schools and community health centers in an effort to increase access to mental health services and build the capacity of partner organizations (Watts & Buckner, 2007).

All thirteen schools in the sample were partnered with CHNP at the time of data collection, the majority since the program’s inception in 2002. The schools included three K-5 schools, six K-8 schools, and four high schools. All 13 are in an urban school district; 12 are within the district system, and one is independent. The schools all serve a high percentage of students who qualify for free lunch (68%), students with demonstrated special needs (18%), and students of color (86%) (Massachusetts Department of Education, 2007).

Participants

In both years of this research, the measure was distributed to all administrators, teachers, and student support staff in each school, with the exception of those for whom there is no expectation they would play a role in addressing students’ mental health. The instrument was included as part of CHNP’s quality improvement activities. We explained to school staff that participation was completely voluntary and that their responses would be anonymous. Most often, the survey was administered during a faculty meeting, and all those present were invited to complete it; the average response rate was 83%.

There were 311 respondents in phase 1; the sample in phase 2 included 400 respondents: 265 teachers, 30 student support personnel, 19 administrators, and 86 other school staff. The typical respondent had been in the same school and in the same role for an average of eight years, though this ranged from one year to more than thirty. On average, administrators and student support staff had more years of experience in their schools (M = 12.1 and 12.3, respectively) than teachers (M = 7.8). No other identifying information was collected about individual respondents.

Individual respondent scores were calculated into one aggregate score for the school. This method follows in the tradition of ecological measurement that uses individual data at the setting level (Revenson et al., 2002). This includes tools that capture individual-level perceptions to draw conclusions about a shared environment, such as neighborhoods (Buckner, 1988), schools (Moos, 1979), or classrooms (Moos, 1979; Trickett & Moos, 1973).

A subsample of these 400 respondents was recruited to complete the SMHCI a second time for the purpose of examining test–retest stability. Those interested in participating were asked to provide their contact information on the original paper version of the survey. This smaller group of respondents who volunteered were then mailed a second copy and/or emailed a link to an online version of the SMHCI approximately one month after initial completion. Of those who indicated interest in participating in the test–retest condition, a total of 79 completed the survey a second time; the response rate was 46%. To preserve respondents’ confidentiality, a unique ID number was created to match the two surveys; all identifying information was captured on a cover sheet that was removed from the survey prior to data entry.

Because there are no existing measures of a school’s mental health capacity as we define it, we had to seek alternate routes to gathering information about the instrument’s validity. To better establish the instrument’s criterion-related validity, six “expert” raters were recruited who had in-depth knowledge of mental health-related work across the 13 schools included in the sample. This group included two educational psychologists and four school-based clinicians who were affiliated with CHNP, but blind to the aims of the study, the specific items on the instrument and the scores of the schools. Each was asked to rate each of the 13 schools as “high” or “low” in mental health capacity. They were instructed to base their ratings on their perception of the school’s overall ability to address students’ mental health needs and whether the school was “reactive” or “proactive” in its approach to mental health. They were asked to think specifically about what the school had in place at the systems level to respond to crises, recognize problems sooner than later, and integrate prevention into the school.

In addition, the principal, student support coordinator, and school-based clinician were also interviewed in two schools (the highest and lowest scoring schools in the sample) to gather additional information about the instrument’s construct-related validity. Staff in each school was asked the same set of questions about the concrete structures, protocols, and services available in each school.

Results

Item Properties

In phase 1, we examined the properties of the individual items in an effort to refine and shorten the instrument. Each of the original 38 items was analyzed for its variability across schools, test–retest stability, and ability to discriminate between schools rated as “high” and “low” capacity by the expert raters. Two items were eliminated because of their relatively high means across all the schools within the range of total possible scores. Eight were discarded because they did not discriminate between high- and low-capacity schools (as judged by the key informants). More information about these processes is described in the sections below. We also looked at the language of items that appeared to be functioning similarly in an effort to eliminate redundancy and shorten the instrument. As previously mentioned, a new version of the instrument containing 27 items (which can be found in Table 1) was put back into the field in phase two (2007–2008). We report on findings from data collected during this second phase.

Table 1 Items in the “school mental health capacity instrument”
Full size table

Factor Analyses

We first conducted factor analyses on the 27-item scale to examine the instrument’s dimensionality. Factors were extracted using principal axis factoring with iteration. Five factors with eigenvalues greater than 1.00 were retained. There was one relatively large common factor with an eigenvalue of 11.44, and four additional factors with eigenvalues greater than 1. The first factor explained 42.4% of the variance; all together, the five factors explained 65.3% of the variance. Because the underlying factors were presumed to be correlated with one another, an oblique rotation procedure (promax in SAS) was used to interpret the factor structure. As shown in Table 1, all nine of the Intervention items loaded highly (ranging from .43 to .77) on the first factor with none of the remaining items from the two other subscales loading higher than .24. Seven of the nine Early Recognition and Referral items loaded highly on the second factor (.47–.76 with the eighth and ninth item loading .29 and .07). The nine items contained in the a priori conceived prevention/promotion subscale were evenly dispersed across the 3rd–5th factors. The third factor consists of items that relate to staff training as well as student awareness of mental health and developmental concerns, which may also tap into the concept of mental health literacy (Jorm et al., 1997). The fourth factor combines items tapping in mental health promotion activities. The fifth contains items specifically related to prevention activities.

Internal Consistency Reliability and Test–Retest Stability

Table 2 shows the alpha coefficients and test–retest correlation coefficients for the overall 27-item scale as well as the three subscales. The alpha coefficient for the 27-item scale, α = .95, suggests that the overall instrument is highly internally consistent. The individual subscales are also internally consistent, r = .87–.92. These results suggest that it is sensible to conceive of the instrument as predominantly unidimensional.

Table 2 Reliability analyses of subscales and overall instrument
Full size table

Correlation analyses to examine the stability between the two separate administrations of the instrument indicate that the overall 27-item scale is fairly reliable over a period of one month, r = .77. The three subscales are also stable over a one-month period, with test–retest correlation coefficients of .60, .72, and .78.

Validity

Out of the 13 schools in the sample, all six key informants nominated the same three schools as having high mental health capacity and the same three schools as having low mental health capacity. As shown in Table 3, schools rated by the informants as having greater mental health capacity also demonstrated higher scores on the SMHCI; schools rated as having lower mental health capacity also demonstrated lower scores on the instrument. Paired t-tests were conducted on the average subscale scores and the total score for the schools in the high-capacity group and the schools in the low-capacity group. For the total score, there was a significant difference between the scores for the two groups, t(162) = −3.38, p < .001, which provides support for the criterion-related validity of this instrument (subscale t-test values were statistically significant at p < .01).

Table 3 Average SMHCI subscale and total scores for schools rated as having high and low mental health capacity by expert raters
Full size table

As the summary in Table 4 suggests, differences in the predicted manner were also found in data gathered from interviews with principals, student support staff, and clinicians from the two schools with the highest and lowest scores on the SMHCI. The staff in School A, the proactive school, was able to produce a greater number of concrete examples of policies, systems, and activities in place to address mental health. Further, efforts to address mental health were included in the overall operations and organization of School A in several ways. For example, time was set aside at regular faculty meetings for teachers to discuss students about whom they had a concern; they were able to get feedback from each other as well as the Student Support Coordinator both about strategies to support the students in the classroom and resources in the school and larger community. In contrast, the staff in School B conveyed a high level of confusion and miscommunication regarding the presence of systems to refer students with mental health concerns as well as little shared knowledge about policies in place to address problems when they arise. In general, they characterized the school as frequently responding to crises with few systematic attempts to integrate prevention or promotion activities.

Table 4 Results from interviews with key informants in the schools with the highest and lowest SMHCI scores
Full size table

Intraclass Correlation Coefficients

Since our method of inferring a school’s mental health capacity is based on the mean of aggregated individual-level data, it is prudent to determine whether enough homogeneity exists among respondents’ scores within a school to make a school-level attribution meaningful. In organizational climate research, a form of the intraclass correlation coefficient (ICC) (Bartko, 1976; Shrout & Fleiss, 1979) has been used to demonstrate agreement among individuals’ perceptions before averaging scores to produce an organizational-level construct (James, 1982; Joyce & Slocum, 1984). We would interpret a high ICC as a sign of small within-group variance and an indicator of good agreement among individuals (Bartko, 1976).

In this study, an ICC less than .15 would be found for a school if the variation around the school mean exceeded the variation of all respondent scores around the total mean. Such a score would suggest that enough within-school variation exists to look for important subgroup differences in sense of mental health capacity which, if detected, should caution a researcher (or consultant) about making an overall attribution concerning a school’s capacity. ICC values were computed for each of the 13 schools and ranged from .05 to .64. Nine of the 13 schools had ICC values above the threshold of .15, indicating there is sufficient homogeneity in individual-level capacity scores within each of these nine schools to warrant making inferences about mental health capacity at the school-level of analysis. Four of the schools had ICC scores below this threshold, making it important to emphasize the substantial variation in ratings that exists among staff members within each of these schools when reporting summary results from the SMHCI.

Instrument Properties

A school’s overall mental health capacity score is calculated by summing each of the respondents’ scores on the 27 items and then taking the average of these scores. The theoretical range for the overall school mental health capacity score (at both the individual- and school-level of analysis) is 0–81; a mean summed score can be calculated for each of the three subscales as well. We found substantial variation in schools’ mental health capacity scores, and there were no “floor or ceiling effects” wherein any school was rated at the lowest or highest end of the instrument. The range in actual total scores for these 13 schools was from 38.1 to 60.7. As Table 5 shows, the average (M = 46.2) and median (M = 48) mental health capacity scores are approximately in the middle of the range of the total possible scores on the instrument (0–81).

Table 5 Mean, standard deviation, and range for schools on the SMHCI (N = 13)
Full size table

Discussion

In this paper, we describe the development and provide preliminary psychometric properties of a new instrument, the “School Mental Health Capacity Instrument” (SMHCI), which is designed to measure the school-level construct of mental health capacity, or a school’s ability to school to address student mental health. The SMHCI appears to provide a way to quantify this notion of mental health capacity. Further, results help locate where a school is along a continuum from reactive to proactive. The degree to which a school is proactive is related to the policies, procedures, and programs a school has in place to address the mental health needs of students.

Based on our preliminary analyses, the SMHCI looks to be a highly reliable scale, based on analyses of its internal consistency as well as its test–retest stability. The construct of “mental health capacity” appears to be predominantly unidimensional. In general, the schools in this sample seem to be consistent in the way they address each of the three categories captured by the SMHCI, namely intervention, early recognition and referral, and prevention and promotion. In other words, a school that is fairly reactive in its approach to intervention, for example, is likely to also be somewhat reactive in how it addresses early recognition and referral; schools are not likely to be highly reactive in one area and highly proactive in the other two. Schools seem to distribute their efforts to address mental health fairly evenly across the three domains. While there is not strong empirical evidence to support the three subscales as capturing highly distinct constructs, as we discuss later on, we believe the subscales have practical value when using the instrument.

Evidence for the criterion-related validity of the instrument was found by comparing external ratings of schools with demonstrated scores on the instrument. Also, qualitative evidence of a school’s mental health capacity was also gathered from key informants within two schools. We intend to continue validation efforts of the SMHCI by exploring the extent to which other measurable indicators, such as quality of mental health services, school-level indicators of academic performance, or perceptions of school climate, are associated with schools’ scores.

Limitations and Future Research

One limitation with the current study is the response rate for the test–retest condition; out of those participants who volunteered to complete the instrument a second time, approximately half ended up participating. We recognize that there may be some bias in the respondents who chose to answer the survey the second time. One possibility for the low response rate may be the majority of respondents, who were teachers, may not have received the information in a timely manner as the one-month test–retest period was during the school summer vacation. In the future, data collection efforts could be timed so they do not overlap with school holidays. Additional efforts could be also taken to outreach to those respondents who volunteered to participate.

Similarly, the relatively small sample size of schools limits us from examining relationships at the setting level. For example, we are not able to speculate about relationships between a school’s mental health capacity and student-level outcomes, though we might expect students in proactive schools to demonstrate more positive mental health outcomes. We are also limited in our ability to discern how the 13 schools in this sample compare with other schools within or across the district, state, or country; in part, these schools are not a representative sample because of their long-standing relationship with the CHNP program. We would expect there are various other factors, both internal and external to a school, that help account for its overall mental health capacity. A larger sample of schools could help pinpoint key organizational characteristics that may contribute to a school’s ability to be proactive in addressing student mental health issues. More extensive representative data would also allow us to generate norms for the instrument, enabling comparisons within or across schools with certain shared characteristics, such as number of students or level of need. At this juncture, the mental health capacity scores we report from these 13 schools should be considered a very preliminary indication of the range in mental health capacity scores across schools in the United States as well as the average score to be found.

While the SMHCI is designed to quantify a school’s mental health capacity at a point in time, it was not developed with the intent of determining why a school has a particular degree of mental health capacity. Reasons that account for a school’s mental health capacity are undoubtedly related to various factors, both internal and external to the school. For example, the amount of available resources is of critical importance to a school’s capacity. With few resources in place, it may be more difficult for a school to adopt a more proactive stance. In contrast, schools that are well resourced may experience fewer crises and feel more able to focus on proactive activities, such as programming that addresses conflict resolution, peer mediation, or the promotion of social skills. Further examination of this issue would require collecting additional information outside the scope of the SMHCI.

Future research could also further examine the evidence of validity, particularly in terms of the instrument’s convergent and discriminant validity. It would be useful to test, for example, whether there are associations between mental health capacity and other constructs, such as the quality of mental health services provided in a school. For example, it would be helpful to examine whether schools’ capacity to address mental health is related to their adherence to the principles of best practice for school-based mental health services (Weist et al., 2005). Similarly, research could explore the relationship between a school’s mental health capacity and its implementation of a positive behavioral support program, such as PBIS. We would expect that schools with higher quality mental health services or more extensive integration of positive behavioral support would also demonstrate greater mental health capacity; though the constructs are distinct, they are likely related to each other.

Further, research could explore the extent to which school mental health capacity is associated with the “mental health literacy” of staff or students. Mental health literacy generally refers to individuals’ knowledge of specific mental health problems, such as the symptoms of depression, as well as their beliefs about the efficacy of treatment (Jorm et al., 1997). It may be that more proactive schools also have higher rates of literacy around mental health.

Implications for School Mental Health

The development of a means to ecologically assess and quantify school mental health capacity offers the potential for fruitful new avenues for research and also provides a useful tool for helping schools better address the mental health needs of students. First, the instrument can help describe a school’s current functioning around mental health. The SMHCI can also provide important information for community mental health partners looking to collaborate in schools. The context of service delivery is important in terms of its “readiness” to receive an intervention (Flaspohler, Duffy, Wandersman, Stillman, & Maras, 2008). In addition, individual-level mental health outcomes are related to the organizational climate in which the program is embedded (Glisson & Hemmelgarn, 1998; Glisson et al., 2008). Future research could examine the relationship between schools’ mental health capacity and the effectiveness and sustainability of school-based mental health programs.

On a broader scale, the results could be used as a needs-assessment. Such information could help detect those schools that might benefit from a capacity-building intervention as well as inform decisions about how to most effectively coordinate the allocation of existing resources with efforts for new programs or partnerships. Results could also pinpoint best practices from schools that demonstrate a more proactive approach to addressing mental health issues.

The SMHCI can be used as a pre–post or time-series measure of the school-level analysis for research and evaluation purposes, particularly when evaluating the impact of mental health interventions that focus on the school as the “client.” Over time, or after a “capacity-building” intervention, a school would begin to move toward the proactive end of the continuum. To help build its mental health capacity, a school may elect to work with an outside consultant as a way to facilitate a change process. Such a consultant would work with the relevant stakeholders in a school to help them develop and implement a plan for building mental health capacity. The results of the SMHCI, then, can be helpful not only in pointing to areas to target, but also the items can serve as a conceptual roadmap. A consultant could examine the items to identify specific things needed to be put into place to build additional capacity. For example, within the area of early intervention, a consultant might help form a group to be responsible for gathering referrals about students in need of mental health services and help the group develop a process.

When the results of the instrument will be used as part of a comprehensive effort to build mental health capacity, we recommend that all school staff complete the survey. To launch a change process across the school, engaging school staff from the beginning will help generate the greatest amount of “buy-in.” It may also be helpful to share the results with staff so that there is shared understanding about what efforts are being undertaken in the school. Likewise, it could be useful to look at the results for various groups of staff members. The presence of differences across teachers and administrators, for example, may be meaningful to consider when trying to launch a change initiative. In addition, efforts should include the perspectives of other key stakeholders, such as parents and students, who have critical perspectives on the school.

In conclusion, schools are likely to remain an important setting for addressing the mental health needs of children and adolescents. Schools that can respond in a proactive manner are likely to function better and be more effective in serving the psychosocial and educational requirements of their students. The School Mental Health Capacity Instrument provides mental health professionals and educators with a new and potentially innovative way to assess and quantify the construct of school mental health capacity.