Introduction

Early childhood disruptive behavior disorders (DBDs), including oppositional defiant disorder (ODD) and conduct disorder (CD), are a public health problem (American Psychiatric Association, 2013; Burt et al., 2018). They are the most common mental health problem among American youth 7 years old and younger, and if not properly managed, are predictive of lifetime psychological problems with significant personal, societal and economic burdens (Lochman et al., 2017; Rivenbark et al., 2018; Wertz et al., 2018).

Major goals in clinical research to effectively manage early childhood DBDs are to develop treatment models that improve problematic behaviors and minimize risk factors of poor outcomes (Muratori et al., 2017; Yu-Lefler et al., 2023). These models need to be based on real world evidence to learn about their implementation and how to support their effective ongoing delivery in community-based settings (Gatti et al., 2019). To date, parent management training (PMT) is the treatment model with the most robust empirical evidence base to manage childhood DBDs (Chorpita et al., 2011; Dishion et al., 2011; Eyberg et al., 2008; Forehand et al., 2013; Helander et al., 2022; Johnson & Katz, 1973; Michelson et al., 2013; van Aar et al., 2017; Weisz & Gray, 2008). However, a large gap still remains between the knowledge gained through empirical research and clinical practice (Chorpita et al., 2005; Garland et al., 2013; Herschell et al., 2006; Michelson et al., 2013; van Aar et al., 2017; Whipple & Lambert, 2011).

PMT is commonly provided within private outpatient clinics in the United States (Garland et al., 2013; Herschell et al., 2006; van Aar et al., 2017). The demands of PMT have been acknowledged (Bornheimer et al., 2018; Gopalan et al., 2010), as the treatment model’s success depends on parent/guardian(s) (henceforth referred to as “parents”) consistently working with their child’s clinician to develop practical skills that enhance positive behaviors in their children and reduce undesirable ones across multiple settings (Forehand et al., 2013; Green et al., 2019; Yu-Lefler et al., 2021). Only by attending frequently and regularly can parents receive timely feedback and guidance to correctly implement effective management approaches and ensure optimal outcomes (Gopalan et al., 2010; Lindhiem et al., 2014; van Aar et al., 2017).

Despite the labor requirements, very little research has examined the treatment trajectory of PMT implemented within U.S.-based private practices for early childhood DBDs (Lindhiem & Kolko, 2010, 2011). Understanding the shape and timing of symptom reduction as a function of PMT treatment engagement for early childhood DBDs provides essential information to clinicians and parents on what is working and how to calibrate expectations (Bybee et al., 2007; Lambert et al., 2005; Lindhiem & Kolko, 2010). To our knowledge, only one U.S.-based pragmatic trial investigated the trajectory of childhood DBD behavioral outcomes by each PMT appointment session attended in a private outpatient clinic (Lindhiem & Kolko, 2010). The trial used parental reports on a custom-made measure for childhood and adolescent ODD/CD (known as the Weekly Report of Behavior Problems [WROB]) to compare appointment-by-appointment changes in behavior problem levels between community/in-home delivery with clinic delivery of parent management training among 6-11-year-old children with moderate to severe DBDs. The study’s clinical treatment effect by the last appointment (which was also the treatment’s maximum effect) was a 50% reduction from baseline behavior problem levels, and required an average of 23 appointments to achieve (Lindhiem & Kolko, 2010). However, it is unknown how much these findings generalize to younger children with DBDs treated with PMT in U.S.-based private practices (de Jong, 2016; Yu-Lefler et al., 2022).

Additionally, very little has been studied regarding risk factors encountered in U.S.-based private outpatient settings that are associated with the child’s or parent’s progress in a PMT program. Existing observational studies on American children with DBDs treated in U.S. mental health care systems suggest baseline behavior severity and complexity along with comorbidity of other Diagnostic and Statistical Manual − 5 (DSM) diagnoses may predict slower progress or more modest treatment gains (Mueller et al., 2010; Wilkie et al., 2018). Patient age, sex, race/ethnicity, and the experience level of their clinician may also introduce disparities in behavioral change trajectories (Mueller et al., 2010; Wilkie et al., 2018). Of note is that these observational studies were based on patients who utilized public mental healthcare facilities and intensive in-home services, and thus, were older (11 to 17 years old when admitted for services) and had more complex mental health profiles. Existing studies conducted on younger American children treated in private, outpatient settings identified social or familial risk factors – e.g., the child’s insurance type, his/her primary parent’s age and level of stress, the disposable income and number of siblings in their household, and if their siblings also required treatment – associated with the patient family’s ability to engage in PMT, but did not directly study how these factors are associated with behavioral outcomes (Crockett et al., 2020; Yu-Lefler et al., 2023).

Because of the significant knowledge gap regarding the ‘real world’ effectiveness of PMT for early childhood DBDs within U.S.-based private outpatient practice, we have undertaken a study using data from a U.S.-based private outpatient behavioral clinic specializing in PMT for childhood behavioral problems. We investigated the trajectory and overall association of DBD behavioral outcomes as a function of PMT treatment engagement, which similar to Lindhiem and Kolko’s 2010 trial, was defined as appointment attendance (in this case, total and consistent attendance). We also investigated demographic, clinical and service risk factors associated with DBD behavioral outcomes within this PMT program. Based on results from Lindhiem and Kolko’s 2010 trial and pilot findings from this clinic (Yu-Lefler et al., 2019), our a priori hypotheses are that reaching optimal DBD behavioral outcomes requires consistent attendance, and that treatment may need up to 20 appointments to reach maximum effect before attenuating. Based on past observational study findings (e.g., Crockett et al., 2020; Mueller et al., 2010; Yu-Lefler et al., 2023), we also hypothesize that the child’s baseline clinical profile, assigned clinician characteristics, and sociodemographic factors will be significantly associated with his/her ability to reach optimal outcomes.

Methods

The setting was an academically affiliated, private outpatient behavioral clinic of a large behavioral health pediatric hospital in the U.S. mid-Atlantic region. The clinic was an American Psychological Association accredited training program, where licensed doctoral-level staff psychologists supervised all therapy conducted by masters-level doctoral interns (in the last year of their PhD or PsyD program) and postdoctoral fellows (supervised training after earning their PhD or PsyD). Interns treated patients on Medicaid exclusively, whereas postdoctoral fellows and staff could treat patients with any insurance-type. The Institutional Review Boards of the pediatric hospital and its affiliated university approved this study.

The clinic provided on-site behavioral assessment and PMT at four urban and suburban locations to over 2,000 typically developing children with significant behavior problems every year. PMT at the clinic was a standardized protocol adapted from principles on parent training for behaviorally-disordered children (e.g., Eyberg, 2013; Kazdin, 1995). The initial three appointments consisted of parents learning PMT behavior principles and working with the clinician to identify the child’s target behavior problems and functioning, appropriate behavior modification strategies, and treatment goals (Yu-Lefler et al., 2021). Clinicians also provided parents with metrics to monitor the child’s and family’s progress toward treatment goals. As needed, clinicians documented any cultural accommodations they made for the patient family. While treatment plans were tailored to each patient and family, clinicians provided all parents with psycho-education on key components of behavior analysis (i.e., antecedents, behaviors, and consequences) and routinely taught common strategies based on behavior analysis principles (e.g., differential attending, time-out, and behavior incentive charts). Follow-up appointments consisted of the clinician and parents reviewing progress toward treatment goals and adjusting strategies, with parents being able to generalize their skills to non-clinic settings as an essential treatment milestone. Each appointment was 60 to 90 min in duration and occurred at a weekly or biweekly cadence. After parents generalized PMT strategies to different settings and the child’s behavior problem appreciably decreased, the clinician recommended monthly maintenance appointments to ensure sustainment of treatment gains.

The clinic integrated written versions of the PMT protocol into both their orientation training materials for new clinicians and patient progress note templates. Clinicians referenced the protocol when assessing each patient family’s progress at each appointment and reviewing challenges to successful treatment during weekly or biweekly clinical team meetings. If the child required services outside the clinic’s scope of practice, the clinician provided referrals for additional or alternative providers. As needed, clinicians coordinated care with the child’s school or the state’s social services to better ensure the child received suitable care management and social support for his/her needs.

Study Population

Over 80% of the clinic’s patient population was 7 years old and younger. 94% of patients had mothers as the primary parent engaged in PMT. This study focused on 4–7 year-old patients who had treatment courses beginning during the period July 1, 2016 to June 30, 2018, attended at least two appointments during their treatment course (intent-to-treat), and had DSM diagnoses of childhood-onset oppositional defiant disorder (ODD) and/or conduct disorder (CD) that included at least one of the following target behaviors: aggression, anger management, disruptive behavior, non-compliance, property destruction, or tantrums. Patients with multiple treatment courses only had their first treatment course included. We excluded patients younger than 4 years old, as they accounted for less than 10% of the patient population and often did not have a clear diagnosis of ODD. The final analytical sample consisted of 1,211 patients who represented 80.6% of all 4–7 year-old patients at the clinic with intent-to-treat treatment courses in the study period. All included patients completed their treatment course at the time of analysis.

Data Sources

We collected retrospective data from the clinic’s measurement feedback system on each patient’s behavioral outcomes, appointment attendance, assigned clinicians, demographics, and baseline clinical assessments. Over the past 12 years, clinic leadership, administrators, and clinicians worked iteratively to design, develop, and deploy this system to improve during-treatment feedback and therapeutic engagement among patient families as well as to assess end-of-treatment outcomes across the clinic’s patient population. This system incorporated administrative data (child/parent demographic, appointment, and service information) and baseline clinical assessments collected as part of “usual care” with appointment-by-appointment behavior data to provide clinicians and clinical leadership with timely, customized reports on individual patients’ treatment progress and population-based clinical outcomes. Clinicians and leaders use the data to make improvements to treatment plans and clinical programs to better address patient needs and optimize outcomes. Further details regarding this system can be found in earlier publications (e.g., Yu-Lefler et al., 2019, 2022). Public census data provided further information regarding the patient family’s disposable household income and travel distance. All study variables and their descriptive statistics are summarized in Table 1.

Table 1 Distribution of variables (N = 1211)
Full size table

Dependent Variable

Similar to Lindhiem and Kolko (2010), we used a brief measure developed, validated, and used at the clinic to assess childhood DBDs. At each appointment during the treatment course, the primary parent provides a rating to the clinician that best describes each of their child’s behavior targets in the past week using the “Behavior Rating Scale” (BRS), a 0–10 Likert scale (“severe” [8–10], “moderate” [4–7], or “mild” [0–3]). For children with multiple target behaviors, a BRS rating was assessed separately for each target and began when the target first presented during the treatment course. The BRS rating by the last appointment (or average BRS rating if multiple targets) was used to indicate clinical outcomes. For this study, a mild BRS rating across all DBD behavioral targets by the last appointment was criterion for the child reaching optimal DBD behavioral outcomes. Prior research conducted the clinic have demonstrated the value and validity of parent-reported Likert ratings against ‘gold standard’ target behavior counts by trained observers (Becraft et al., 2023). Multiple internal analyses conducted at the clinic have also demonstrated high correlations (r > .70, p < .01) between last appointment BRS ratings (both individual and average) with the Child Behavior Checklist (CBCL) (Achenbach, 1991) and the Symptoms and Functioning Severity Scale (SFSS) of the Peabody Treatment Progress Battery (Athay et al., 2012).

Independent Variables

We constructed two variables to reflect a patient’s PMT treatment engagement: (1) total attendance and (2) consistent attendance. Total attendance was the number of attended appointments during the treatment course, which included originally-scheduled and rescheduled appointments for which families showed. This count does not include originally-scheduled or rescheduled appointments that patients missed or cancelled without rescheduling prior to the end of their treatment course. Consistent attendance was the attendance of 75% or more of all originally-scheduled or rescheduled appointments. This was slightly above the mean percentage of attended appointments at the clinic (74.0%), which clinicians considered as the minimum threshold of treatment engagement for successful outcomes (Yu-Lefler et al., 2021).

We also included demographic, service, and clinical factors indicated by past literature that are associated with children’s DBD behavioral outcomes (Drugli et al., 2010; Drugli, LarssonDrugli et al., 2010a, b; Forehand et al., 2014; Gopalan et al., 2010; Helander et al., 2022; Larsson et al., 2009; Reyno & McGrath, 2006; Yu-Lefler et al., 2021, 2019). Demographic factors consisted of patient age, sex, race/ethnicity, primary parent age, insurance-type, number of siblings in the patient’s household, the patient family’s disposable household income, and travel distance to the clinic. We used census block group in lieu of zip code for each family’s disposable household income, as it is a more accurate estimation of the neighborhood’s socioeconomic characteristics. We geospatially mapped the patient’s home address to a specific census block group and its respective U.S. Census 2020 average disposable household income using ArcGIS Pro, Version 2.4.2.(Esri Incorporated, 2019). ArcGIS Pro also provided the travel distance from the patient’s home to the clinic site by geocoding both the site and home address, and then calculating the shortest driving distance without tolls (in miles) between the two points.

Service factors consisted of the patient’s assigned clinician(s), clinician experience levels (doctoral intern, postdoctoral fellow, licensed staff psychologist), duration of the treatment course (in days) at the clinic, whether the patient received additional services at the hospital during the treatment course, and whether sibling(s) simultaneously attended treatment at the hospital. We included the first appointment’s clinician experience level and the overall number of assigned clinicians, as patients assigned to more than one clinician had those with same experience level.

Clinical factors consisted of patient’s behavioral severity and clinical complexity, and the parent’s emotional and functioning levels. The patient’s first appointment average DBD BRS rating indicated baseline behavior severity. Additionally, Behavior Assessment System for Children, Second Edition (BASC) and Parenting Stress Index – Short Form 3 (PSI) subscales (i.e., externalizing problems, internalizing problems, adaptability, parental distress, parent-child dysfunctional interaction, and difficult child) completed by the primary parent during the first three appointments indicated the family’s baseline emotional and functioning levels. The BASC and PSI are widely-used and highly reliable assessments among parents of children with behavioral disorders for application to diagnostic issues across diverse patient populations (Abidin, 1995; Altmann et al., 2019). Internal consistency reliability was also high in the present study population, as alpha coefficients for all subscales exceeded 0.80. We used the recommended cut-offs for clinically significant problems: BASC internalizing and externalizing problem T scores of 70 or higher, BASC adaptability T score of 30 or lower, and PSI subscale scores at the 85th percentile or higher (Abidin, 1995; Reynolds & Kamphaus, 2004). Lastly, we used the number of DBD target problems and comorbid non-DBD diagnoses (from most to least prevalent: sleep-wake, mood/anxiety, feeding and eating, non-suicidal self-injury, and hair pulling [trichotillomania] disorders) identified throughout treatment to determine the child’s clinical complexity.

Data Analysis

We conducted all analyses using STATA/IC, v. 13. Descriptive analyses examined distribution and data missingness in all variables. Only one patient out of 1211 (0.08% of the sample) was missing primary caregiver age; however, more than 300 patients (> 24.8%) were missing baseline BASC and PSI subscale scores. Descriptive analyses also illustrated the trajectory of proportion of patients reaching optimal DBD behavioral outcomes as a function of total attendance, and the association between reaching optimal outcomes and consistent attendance over the treatment course.

Regression analyses assessed the associations between reaching optimal outcomes (i.e., mild BRS rating [≤ 3] across DBD target behavior problems by the last appointment) and the independent variables. Independent variables were added progressively, with model and variable selection determined by Variance Inflation Factor (VIF) and Akaike information criterion (AIC) that checked for, respectively, multi-collinearity and best fit. We assessed the associations of total and consistent attendance with reaching optimal outcomes in separate regression models, due to the two independent variables being conceptually related and highly collinear (VIF > 6). In both models, exclusions included treatment course duration due to high collinearity (VIF > 10) with total and consistent attendance, and clinician experience level in lieu of the patient’s insurance-type due to perfect collinearity for doctoral intern-Medicaid assignments. Both models retained all other independent variables. To evaluate our hypothesis of outcomes attenuating upon reaching maximum effect, we tested both linear and parabolic equations to evaluate which has the best fit for the focal relationship between total attendance and reaching optimal outcomes, in which AIC identified the parabolic equation had the best fit. As all patients in the analytical sample were assigned to 44 unique clinicians, we also assessed for clustering of patient DBD behavioral outcomes by clinician using intraclass correlation (ICC). To account for missingness in primary caregiver age and the BASC and PSI subscale scores, we used multiple imputation with 10 sets of imputed data sets for missing values, using continuous scores for more accurate imputed values (Austin et al., 2021). The final models use logistic regression with imputed data. We also compared the final model results to models without imputed data, multi-level mixed effects regression models that adjusted for clustering of patient outcomes by clinician, and linear models for total attendance and reaching optimal outcomes.

Results

Table 1 shows descriptive characteristics for all the variables. Over the treatment course, 40% of patients reaching optimal DBD behavioral outcomes. The average treatment course was 161 days (SD = 159), during which patients attended, on average, 9–10 appointments. Over 50% of patients had consistent attendance during their treatment course. Most patients were assigned to one clinician, with almost 80% of clinicians being Masters-level or post-doctoral trainees. Almost 90% of patients began treatment with severe behavior problem levels. More than 80% of patients were treated for only disruptive behavior issues and only one DBD target problem. 66% of children began treatment having clinically significant externalizing and adaptability problems on the BASC, and more than 50% were in the clinically significant range on the difficult child subscale (DC) of the PSI. 33% of patients received additional services in the hospital system. Just over 10% of patients had siblings also concurrently in treatment. Patients were, on average, 5 years old and predominantly male. Almost half of the patients were White. Most non-White patients were Black or multi-racial. Almost all patients had commercial, Medicaid, or military (Tricare) insurance plans. The patients came from households where the primary caregiver averaged 36 years old (SD = 7.4), with one to two siblings. Annual disposable household incomes averaged $64,000 (SD = $24,000). Families lived, on average, 20 miles (SD = 87.9) from the clinic site.

Figures 1 and 2, and 3 illustrate how DBD behavioral outcomes changes as a function of PMT treatment engagement. The proportion of patients who reach optimal DBD behavioral outcomes (BRS rating ≤ 3 across DBD targets by the last appointment) increases with each additional appointment attended, reaching over 50% of patients after 10 appointments and maximizing at 75% of patients by 24–29 appointments (Fig. 1). To compare these effects with that of Lindhiem and Kolko’s 2010 trial, we present an alternative figure displaying the average change in DBD behavior problem level from initial to last appointment by total attendance (Fig. 2). The alternative figure shows patients achieved over a 50% reduction in their DBD behavioral symptomology by 10 appointments and a maximum treatment effect of 70% reduction by 24 to 29 appointments, which is better than the outcomes observed within the 2010 trial. Additionally, the proportion of patients reaching optimal outcomes doubles for those with consistent attendance compared to those who did not (Fig. 3).

Fig. 1
figure 1

Proportion of patients reaching optimal DBD behavioral outcomes by total attendance

Full size image
Fig. 2
figure 2

First-to-last appointment change in DBD behavior problem rating by total attendance

Full size image
Fig. 3
figure 3

Proportion of patients reaching optimal DBD behavioral outcomes by consistent attendance

Full size image

Table 2 presents factors significantly associated with reaching optimal DBD behavioral outcomes from the best fit logistic regression models. Models 1 and 2 assessed the associations of, respectively, total, and consistent attendance with reaching optimal outcomes. Like the trends in Figs. 1 and 3, DBD patients had increasingly higher odds of reaching optimal outcomes for each additional appointment attended (odds ratio [OR]: 1.25, confidence interval [CI]: 1.18, 1.31), and three times higher odds of reaching optimal outcomes by their last appointment if they had consistent attendance (OR: 3.21, CI: 2.45, 4.21). Model 1 also indicates an inverse parabolic relationship between reaching optimal outcomes for each exponential increase in attended appointments (OR: 0.997, CI: 0.995, 0.997), meaning that the odds of reaching optimal outcomes increase until a maximum effect is reached, after which odds decrease. In addition to PMT treatment engagement, both models indicate patients had higher odds of reaching optimal outcomes if they had military insurance (vs. Medicaid) (OR range: 1.63, 1.73, CI range: 1.03, 2.73); and lower odds if they had a higher first appointment behavior problem rating (OR range: 0.86, 0.89, CI range: 0.78, 0.97), and siblings concurrently in treatment (OR range: 0.61, 0.66, CI range: 0.40, 0.98). Model 1 indicates lower odds of reaching optimal outcomes if patients had comorbid non-DBD diagnoses (OR: 0.67, CI: 0.47, 0.96). Model 2 indicates higher odds of reaching optimal outcomes if patients had more than one DBD target problem (OR: 1.85, CI: 1.14, 2.57), and more than one assigned clinician (OR: 1.49, CI: 1.11, 1.99). Insignificant factors were patient age, sex, race/ethnicity, primary parent age, number of siblings in the patient’s household, the patient family’s disposable household income, travel distance to the clinic, patient received additional hospital services during treatment course, and the BASC and PSI subscales. These results remain similar in linear models between total attendance and reaching optimal outcomes, models without imputed data, and multi-level mixed effects models adjusting for clustering of outcomes by clinician (ICC = 0.04).

Table 2 Significant predictors of reaching optimal outcomes for disruptive behavior disorders
Full size table

Discussion

This study provides novel insight into (1) the trajectory of early childhood DBD behavioral outcomes as a function of PMT treatment engagement in a U.S. private outpatient clinic, and (2) how these behavioral outcomes may be affected by demographic, clinical and service risk factors. Similar to the trajectory found in Lindhiem and Kolko’s 2010 trial, the percent of patients who reached optimal DBD behavioral outcomes increased with each additional appointment attended, reaching a maximum effect (75% of patients reaching a mild level of behavior by their last appointment, or alternatively, 70% behavior problem reduction from the initial to last appointment) around 24 appointments before attenuating. Additionally, patients who had consistent attendance throughout the course of treatment were more likely to reach optimal behavioral outcomes. Notably, the present study had a higher maximum treatment effect (70% compared to 50% behavior problem reduction) for a similar number of appointments attended and reached the 2010 trial’s maximum effect with less appointments (10 appointments). These findings may be due to the younger patient population (4–7 year olds) compared to that within the 2010 trial (6–11 year olds), as younger youth often have less impairments when entering treatment and faster improvement rates (de Jong, 2016; Mueller et al., 2010).

Controlling for PMT treatment engagement, several demographic, clinical, and service risk factors had significant associations with early childhood DBD behavioral outcomes. As hypothesized, patients with more severe levels of baseline behavior problems, co-occurring non-behavioral problems, and siblings concurrently receiving treatment were less likely to reach optimal outcomes than less complex patient cases. These findings are similar to results found in PMT efficacy trials, namely that DBD behavioral outcomes may be more modest for patients from low-income backgrounds, with greater initial behavior severity and/or clinical complexity, and siblings concurrently in treatment (Drugli et al., 2010a, b; Forehand et al., 2014; Larsson et al., 2009; Reyno & McGrath, 2006). Patients who were on Medicaid had worse outcomes than patients who had military insurance. Resource differences may explain these behavioral outcome disparities. Past studies on treatment retention in children’s mental health care suggest military families likely have a support system and culture that reinforces persistence in finishing treatment, which increases the odds of achieving better behavioral outcomes (Crockett et al., 2020; Harpaz-Rotem et al., 2004). Comparatively, families on Medicaid may lack similar resources to help them persist in treatment and achieve similar positive outcomes (Charles et al., 2011; Forehand et al., 2011; Harpaz-Rotem et al., 2004; Parks & O’Malley, 2016; Reid et al., 2016; Reyno & McGrath, 2006). Interestingly, patients increased their odds of reaching optimal outcomes if they had more assigned clinicians and more DBD target problems. However, this may reflect a longer treatment course which promotes a positive outcome and leads to more clinicians being assigned due to training rotations as well as identification of additional disruptive behavioral issues (Helander et al., 2022; Michelson et al., 2013; Yu-Lefler et al., 2021, 2019).

The results have implications to clinical operations and standards of practice for PMT. First, based on the treatment trajectories, accessing PMT when children are younger may be essential to maximizing treatment effectiveness for DBDs (Chacko et al., 2016; Gopalan et al., 2010; Lindhiem & Kolko, 2010). This is particularly important for families from underserved populations or with multiple children displaying behavioral problems but limited ability to attend treatment within usual care settings. Diversifying the delivery of PMT – such as through internet-based therapy, community-based mobile clinics, and integration within primary care practices – may provide more equitable and timely access points for American families seeking care (Bultas et al., 2017; Fleming et al., 2017; Malone et al., 2020).

Second, this study and the 2010 trial both indicate that many PMT sessions are required to achieve optimal DBD behavioral outcomes within a clinical population. Part of this effect may be due to the measures for behavioral outcomes used in clinical practice, which often must balance reliability and sensitivity to behavioral change with the need for brevity and clinical utility (Lindhiem & Kolko, 2010). While the measures for DBDs used in both studies have been validated against standardized clinical measures, information regarding PMT treatment trajectories could be strengthened by incorporating validated assessments for evaluating childhood DBDs such as the SFSS and the Youth Outcomes Questionnaire (Y-OQ) as part of standard practice to confirm treatment effect (Burlingame et al., 2005; Whipple & Lambert, 2011). Another part of this effect is that families attended maintenance appointments after the child reached optimal outcomes, which extended patients’ treatment courses. Better definitions for “end-of-treatment” that balance the benefits of continuing treatment to maintain gains with discharging when the child reaches optimal outcomes can also help clarify the treatment trajectory (Gopalan et al., 2010; Nelson et al., 2013; Whipple & Lambert, 2011). Additionally, we lack information across multiple clinical settings with diverse early childhood populations treated by PMT to truly know how many PMT sessions are required to achieve optimal outcomes. This is largely due to the lack of systems monitoring children’s outcomes in routine practice (Yu-Lefler et al., 2022). Clinics that treat early childhood DBDs frequently lack data infrastructures to monitor and provide feedback to clinicians regarding the outcomes of the children they treat (de Jong, 2016; Garland et al., 2013; Liu et al., 2019; Yu-Lefler et al., 2022). As reimbursements for clinical services increasingly emphasize performance (Bremer et al., 2008; Stewart et al., 2017), there ought to be parallel initiatives that incentivize and support building such infrastructure across U.S. mental health care systems (Yu-Lefler et al., 2022).

Limitations

This study has several limitations. First, our research was conducted in one private, outpatient PMT clinic for typically developing children, which was part of a single pediatric hospital system. As stated above, the present study’s findings can be strengthened with further research on the treatment trajectories of PMT for early childhood DBDs in additional children’s private, outpatient behavioral clinics from multiple health care systems.

Second, the study used retrospective data to assess outcomes ultimately achieved within a clinical population, which did not inform if there were patients who may have reached optimal outcomes during their treatment course and regressed by their last appointment (i.e., had behavior problems re-intensify). The use of end-of-treatment outcomes was due to data abstraction limitations of the measurement feedback system at the time of the study, which did not allow for aggregated behavioral data across patients by visit number. The resultant observational study design also limits the present study’s ability to determine the temporal relationship between PMT treatment engagement and behavioral outcomes (for example, it is possible that patient families who did not show promising behavioral progress early on were dissatisfied with the PMT approach, and thus, dropped out early or continued inconsistently). So long as the data is available, follow-up studies may consider using measurement feedback data prospectively to investigate how quickly patients first reach an optimal behavior problem level (i.e., time-to-effect), whether these treatment gains were maintained by their last appointment, and what factors are associated with patients’ ability to reach or maintain gains.

Third, the training structure of the clinic had doctoral interns exclusively treating patients insured by Medicaid. As such, this study could not distinguish how much outcomes for Medicaid patients were confounded by clinician experience level. A follow-up study should investigate the relationship between different clinician experience levels and the PMT treatment trajectory among early childhood beneficiaries of Medicaid diagnosed with DBDs.

Fourth, it is unknown if cultural differences between the clinician and patient family could have influenced PMT engagement or behavioral outcomes. A prior study (Yu-Lefler et al., 2021) indicated that most clinicians at the clinic may have been White. Although clinicians did document cultural accommodations made during treatment, follow-up investigations ought to examine how well parents of a different racial/ethnic background from the clinician understood the psycho-education provided, and ascertain how successfully the accommodations addressed family preferences and cultural needs.

Fifth, all patients included in this study received PMT on-site. Since the COVID-19 pandemic, PMT in usual care has rapidly switched to telehealth service delivery. A follow-up study utilizing more recent outcomes data from the clinic’s measurement feedback system that compares treatment trajectories before and after the rapid switch to telehealth-based treatment may help inform policy decisions regarding the utility of and reimbursement for telehealth-based therapy.

Conclusion

Real world evidence on how PMT works in private outpatient care is needed to inform the development of effective treatment models for young, American children with DBD. To generate this evidence pool, clinical practices need systems that routinely monitor treatment progress and provide feedback on factors influencing progress for young patients (Bickman, 2008). When deployed at a population level across child mental health care, the wealth of data that could be available can allow clinical researchers and practitioners to more accurately evaluate the effectiveness of PMT and adjust its delivery across different clinical settings and patient populations (Bickman, 2008, 2012). Increased implementation of measurement feedback systems and clinical research using data from these systems in private outpatient behavioral practices can further ensure U.S. mental health services successfully manage DBDs in early childhood, thereby preventing more costly psychological and societal burdens.