Children and adolescents involved with the juvenile justice system have exceptionally high rates of trauma exposure and victimization. Ford et al. (2006) reported that at least 75 % of justice-involved youth have been witnesses or victims of physical abuse, neglect, or maltreatment. Abram et al. (2004) found that 92.5 % of juvenile detainees experienced at least one trauma, and Ford et al. (2012) found the rates of complex trauma exposure in these youth were three times greater than community peers. Rates of post-traumatic stress disorder (PTSD) in juvenile justice samples range from 4.8 to 52 % (Abram et al. 2004; Steiner et al. 1997; Wasserman et al. 2004; Wood et al. 2002), and even those youth who fail to meet diagnostic criteria for PTSD often report significant clinical impairment related to post-traumatic stress symptoms (Dierkhising et al. 2013). The impact of trauma is complex and far-reaching. In children and adolescents, attention problems, difficulties with impulse control, learning difficulties, negative self-attributions, memory problems, emotional dysregulation, and sleep disturbance are just a few of the more common responses to trauma and chronic stress (Pynoos et al. 1996). The Adverse Childhood Experiences study found a dose-response relationship between the number of adverse childhood events and later health problems, including disrupted neurodevelopment, immunological disease, cardiovascular disease, diabetes and obesity, respiratory disease, and cancer as well as a greater risk of depression, behavioral issues, substance abuse, and suicide (Felitti et al. 1998).

Although high rates of trauma are clearly present in juvenile justice involved youth, there remains a system-level under recognition of the importance and pervasiveness of trauma. The majority of individuals working with justice-involved youth are not mental health clinicians and receive little to no training that addresses the impact of childhood maltreatment and trauma on development, affect, and behavior. There is a real need for trauma-related training for staff working with justice-involved youth. Further, there is a need for valid measures to evaluate the effectiveness of training and identify content areas that may need additional time and focus during the initial training or booster training.

To address the need for trauma-informed training within the juvenile justice system, the National Child Traumatic Stress Network (NCTSN) developed Think Trauma (Marrow et al. 2012), a training curriculum that educates staff about the sequelae and impact of traumatic experiences of justice-involved youth; how post-traumatic stress symptoms impact their cognition and behavior; the impact of trauma on development; vicarious trauma and organizational stress; and ways for juvenile justice workers to cope with their own traumatic experiences. The curriculum was originally intended for staff working with juvenile justice involved youth in residential and secure facilities, but it has also been delivered to probation officers, residential staff in mental health programs, judges, law enforcement personnel, lawyers, and child welfare workers (M. Marrow, personal communication, 11/20/2013).

The Think Trauma Evaluation Questionnaire (TTEQ) (Richardson et al. 2013), a 45-item questionnaire, was designed to mirror the fundamental topics covered in the Think Trauma curriculum. This questionnaire was created by a NCTSN panel of experts to assess general knowledge and attitudes about post-traumatic stress symptoms, the perceived impact of trauma on delinquency, and attitudes about PTSD with justice-involved youth. The questionnaire is currently being used in multiple facilities, but the psychometric properties of the questionnaire have not been examined. Therefore, this study examined the factor structure and internal consistency of the questionnaire, as well as whether responses varied by the respondents’ sociodemographic characteristics. It was hypothesized that the factor structure would mirror the areas (i.e., trauma and delinquency; trauma’s impact on development; coping strategies; and vicarious trauma; organizational stress; and self-care) in the Think Trauma curriculum. It was also hypothesized that greater educational attainment, medical or psychological training, and recent entrance into the field of juvenile justice would be associated with increased knowledge about trauma and with more trauma-informed attitudes about justice-involved youth.

Methods

Subjects

Study participants were employees at two secure juvenile detention centers in a large, northeastern city participating in Think Trauma as part of a Substance Abuse Mental Health Services Administration (SAMHSA)-funded community implementation project. Participants at both facilities completed the TTEQ before any staff received the Think Trauma curriculum. All active employees were eligible to participate in the training (N = 449). Of these, 324 (72.2 %) staff completed the TTEQ, 163/231 (70.5 %) at Center A, and 161/218 (73.9 %) at Center B. Study participation was approved by the New York University IRB and the New York City Administration for Children’s Services IRB.

Procedures

Administrators at both sites distributed questionnaires to all active employees between February 1, 2013 and March 31, 2013. Participation was voluntary, and detention facility administrators or supervisors did not have access to completed questionnaires. Questionnaires were collected in sealed envelopes by supervisors and subsequently returned to the investigators.

Materials

Sociodemographic characteristics collected included sex, highest educational level, job classification within the juvenile detention facility, years of employment within juvenile justice and prior training in trauma. Educational attainment categories were combined to create a four-level ordinal variable: high school, some college/associates degree, bachelor’s degree, or graduate degree. The 14 original job classifications were recoded to create five categories: administrative/facilities (e.g., manager, administrator, clerk, food service, housekeeping); educators (e.g., teacher, school counselor); group services (e.g., counselor, associate counselor, tour commander); other health worker (e.g., mental health clinician, caseworker, medical doctor or nurse); and special officer/sergeant.

The Think Trauma Evaluation Questionnaire is a 45-item questionnaire designed to assess trauma knowledge and attitudes. Each item is rated on a five-point Likert scale with the following response choices and scoring system: strongly agree (5); inclined to agree (4); undecided/I don’t know (3); inclined to disagree (2); strongly disagree (1). The questions corresponded to the areas covered in the training curriculum including knowledge about the trauma experienced by these youth, the possible sequelae of such trauma, vicarious trauma, organizational stress caused by this work with traumatized youth, and attitudes towards justice-involved youth. Questions included, “Sounds, places, people, smells, images, feelings and memories can all be trauma reminders,” and “I don’t want to give up on these youth”.

Analysis

Subject characteristics and the TTEQ responses were summarized using means and standard deviations for continuous measures and counts and proportions for categorical variables. Participant characteristics for the two juvenile detention centers were compared using the Pearson and Mantel-Haenszel Chi-Square tests. The factor structure of the TTEQ was examined using exploratory factor analyses (EFA). Principal axis factoring was used for factor extraction; Velicer’s minimum average partial test and the scree test were used to select the number of factors to retain. Promax, an oblique rotation that allows the factors to be correlated, was used to facilitate interpretation. Items with a primary loading ≥0.40 and cross-loadings <0.30 were included in subsequent analyses. For factors with nine or more items, the eight items with the highest factor loadings were retained. Factor scores were calculated by taking the mean of the items and multiplying the result by ten, with higher scores indicating greater agreement. Cronbach α was used to assess internal consistency reliability for each factor.

Multivariable linear regression was used to examine the association of subject characteristics with the TTEQ factor scores. Models included juvenile justice center, sex, education, years of employment within juvenile justice, job classification within the juvenile detention center, and prior trauma training. For categorical variables with more than two levels, post-hoc pairwise comparisons were examined if the global p-value was statistically significant at p < .05. PASW Statistics 18 (release 18.0.0, SPSS Inc, Chicago, IL, 2009) was used for EFA, and SAS (version 9.2, SAS Institute Inc., Cary, NC, 2008) was used for all other analyses.

Results

Of 324 questionnaires completed by staff, 296 were included in the analysis. Twenty-eight were excluded due to missing data on the TTEQ (n = 17), or an inattentive response pattern indicating that the staff member had not read the questions (n = 11). Subject characteristics for the 296 participants are shown in Table 1. Overall, 57.3 % of participants were female, nearly three-fourths (70.7 %) had a college degree or higher, and 39.6 % reported working in the juvenile justice system for more than 10 years. Half (50.3 %) of participants were juvenile counselors, and about one-fourth (26.5 %) reported receiving prior training on trauma. Participant characteristics were similar for the two centers (Table 1) with the exception that participants at Center A had more years of experience working in the juvenile justice system compared to participants at Center B (p = .008).

Table 1 Subject characteristics for all participants and stratified by center
Full size table

Results of the exploratory factor analysis indicated a 5-factor, 23-item solution (Table 2): knowledge of trauma (8 items); positive impact of trauma-informed practices on individual staff (4 items); positive impact on the workplace (4 items); awareness of vicarious trauma (3 items); and cynicism (4 items). Standardized factor loadings ranged from Λ = 0.40 on the positive impact on the workplace factor to Λ = 0.84 on the positive impact on individual staff factor. Twenty-two items were omitted from the final factor analysis solution due to a low primary factor loading on all 5 factors (n = 11), a low primary factor loading on the knowledge of trauma factor (n = 6), or high factor crossloadings (n = 5). Descriptive statistics, internal consistency reliability, and inter-factor correlations are shown in Table 3. Cronbach alpha was highest for the knowledge of trauma factor (α = 0.89) and was <0.70 for the positive impact on the workplace (α = 0.68) and cynicism (α = 0.60) factors. Inter-factor correlations were low to moderate; knowledge of trauma and positive impact on the workplace factors had the strongest correlation (r = 0.50).

Table 2 Descriptive statistics and standardized factor loadings (Λ) from the exploratory factor analysis of the Think Trauma Evaluation Questionnaire
Full size table
Table 3 Descriptive statistics, Internal Consistency Reliability (α), and inter-factor correlations of the 5 Think Trauma Factors
Full size table

Multivariable linear regression results are presented in Table 4. Higher educational attainment was associated with higher scores for the positive impact on individual staff factor and the positive impact on the workplace factor, and tended to be associated with higher scores on the knowledge of trauma factor. Job classification was also significantly associated with scores on those three factors. Post-hoc pairwise comparisons showed that group services workers had significantly higher scores than special officers/sergeants on the positive impact on individual staff factor. On the positive impact on the workplace factor, administrative/facilities workers and other health workers had significantly higher scores than special officers/sergeants, and other health workers also had significantly higher scores than educators. Knowledge of trauma factor scores was significantly higher for administrators and other health workers compared to special officers/sergeants and group services workers. Additionally, other health workers had significantly higher knowledge of trauma factor scores compared to educators. Examination of the other sociodemographic measures showed that women compared to men and those with prior trauma training versus those without prior training had significantly higher scores on the knowledge of trauma factor, but these characteristics were not related to scores on the other four factors. Juvenile detention center and years of experience working in juvenile justice were not significantly associated with any of the factor scores. None of the subject characteristics were associated with scores on the awareness of vicarious trauma or cynicism factors.

Table 4 Multivariable linear regression models examining the association of subject characteristics and 5 Think Trauma Factors
Full size table

Discussion

To our knowledge, no factor analyses have been performed on existing PTSD knowledge and attitude measures. Previous instruments have measured knowledge about PTSD risk factors, assessment and diagnosis, management, and treatment as well as attitudes about stress and mental illness; but there is little information about the sociodemographic characteristics related to PTSD knowledge and attitudes. Further, the majority of these instruments were designed for use with specific populations, such as clinicians or military personnel. These questionnaires reflect the population of intended use, focusing on knowledge related to the diagnosis and medical management of PTSD (Mckenzie and Smith 2006; Munro et al. 2004; Ruzek et al. 2012; Samuelson et al. 2014; Weine et al. 2001; Ziegler et al. 2005) and the stigma surrounding mental illness in the military (Gould et al. 2007), respectively.

Initial use of the TTEQ showed it to be reasonable to administer and acceptable to staff with almost 300 individuals across a range of employment positions in two juvenile detention facilities completing it. As hypothesized, the results identified five factors that correspond to material covered in the Think Trauma curriculum: knowledge, impact on staff, impact on workplace, awareness of vicarious trauma and cynicism. The internal consistency of the factors ranged from moderate (cynicism; α = 0.60) to excellent (knowledge; α = 0.89). The knowledge factor is the most robust, with factor loadings ranging from 0.61 to 0.78.

While the knowledge factor had eight items, the remaining four factors consisted of three or four items each. This may have occurred because of the 45 questions, over half (n = 25, 55 %) assessed knowledge, while relatively few items focused on impact on staff (N = 7, 16 %), impact on workplace (N = 3, 7 %), awareness of vicarious trauma (N = 4, 9 %), or cynicism (N = 6, 13 %). The positive impact on the workplace and cynicism factors both had internal consistency reliability <.70, and two of the four factor loadings were <0.50. These data suggest that the validity and reliability of these factors could be improved by including additional items on each factor and/or revising the wording of the items to improve their content validity. Further, an item analysis found that 8 of the 45 items were complex statements with at least two concepts to rate (e.g., “I want to make a difference in the lives of these youth, but am not sure what I can do.”), and 25 items required at least a 10th grade reading level, a potential problem when almost 8 % of the workforce had only a high school education (Doak and Doak 1980; Doak et al. 1996). Revising the items to assess a single concept, adjusting the reading level of the items, and adding items to assess constructs with few items are likely to improve the validity and reliability of the questionnaire.

Factor scores did not vary by juvenile justice center or number of years working in juvenile justice, and with the exception of the trauma knowledge, factor scores did not vary by sex or prior trauma training. As hypothesized, education and job classification were related to knowledge, staff impact and workplace impact factor scores, but these were not related to vicarious trauma or cynicism factors.

These data are not without limitations. While the overall response rate was 72 % and participants completed the questionnaire anonymously, the possibility of nonresponse bias and/or social desirability bias cannot be excluded. Although almost 300 questionnaires were available for analyses, this was a reasonably long instrument (45 items) and there is a general agreement that larger sample sizes produce the most replicable factor analysis results (Costello and Osborne 2005). Issues with a variable number of items measuring each factor, complex statements with multiple parts and the reading level of the statements may have influenced the validity and reliability of the questionnaire. Finally, this study did not assess concurrent validity, predictive validity, or other forms of reliability such as test-retest reliability of the questionnaire.

Conclusion

Regardless of limitations, this initial examination of the TTEQ suggests that the questionnaire is feasible to administer in a large group format, reasonably easy to complete and that the factor structure corresponds to the areas covered in the Think Trauma curriculum. Modest internal consistency and factor loadings, as well as statement specific issues, argue for item revision, creation of new items, and retesting. A reliable and valid measure of trauma knowledge and attitudes can be used to help to identify training needs for a particular facility or population as well as evaluate the effectiveness of the Think Trauma curriculum.

The juvenile justice and child welfare systems are beginning to adopt and integrate trauma-informed approaches as part of their standard practices. Regardless of the system of care, staff training and education will remain an integral part of any trauma-informed intervention. Evaluation of the effectiveness of training curricula is an important step in the development of trauma-informed curricula, and evaluation is not possible without a valid and reliable measure to assess effectiveness. These findings will inform future iterations of the Think Trauma Evaluation Questionnaire and, hopefully, will insure that future forms of the questionnaire effectively measure the impact of the Think Trauma curriculum.