Over the last several years, there has been considerable and increasing interest in evidence synthesis in prevention science. Although systematic reviews and meta-analysis of summary data (Glass, 1976, 2000) remain prominent tools for evidence synthesis (Tanner-Smith et al., 2022), syntheses using individual-level or “raw” data across studies are fast-becoming an important tool across prevention research. The synthesis of individual-level data across studies can complement traditional syntheses by allowing for simultaneous modeling of individual- and study-level predictors (Lubinski & Humphreys, 1996), ensuring consistency in the analysis model across studies, and allowing for increases in model complexity beyond meta-regression models (e.g., Cooper & Patall, 2009). Systematic reviews and summary data meta-analyses retain the advantage of summary data being much more readily available from publications, particularly if the interest is in generalizing the findings of such an evidence synthesis to a larger universe of studies (Tanner-Smith et al., 2022).

Two prominent, and often interrelated, frameworks have emerged in individual-level data synthesis: meta-analysis of individual participant data (MIPD) and integrative data analysis (IDA). While some may argue that IDA and MIPD are synonymous, we argue that they are fundamentally different and serve different roles in a harmonization study. MIPD (Cooper & Patall, 2009; Stewart & Parmar, 1993) was originally touted as an individual data synthesis framework where the primary interest is in characterizing cross-study variability in the intervention effects of interest, much in the way cross-study variability is a primary interest in “traditional” meta-analysis. In contrast, IDA (Bauer & Hussong, 2009; Curran et al., 2008) was originally proposed as a cross-study measurement modeling framework for estimating commensurate scale scores across studies across different developmental periods, as part of an accelerated longitudinal/cohort sequential study where (a) the same item content may not overlap across studies for the same construct and (b) the interest was in making a singular combined inference across studies (and across development). The two approaches have been combined in prevention (Brown et al., 2018), addressing both cross-study variation in intervention outcomes under MIPD and cross-study measurement variation under IDA.

The current special issue of Prevention Science brings together these concepts with the overarching aim of providing an update on current and future directions in IDA, with careful consideration of innovations and applications of these methods to address contemporary issues and research gaps in prevention science. Specifically, the current special issue, entitled “Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science,” includes a first set of papers which showcases evidence synthesis papers from a broad array of areas within prevention that make use of current best practices in data harmonization and individual-level data synthesis and represent substantial leaps in cumulative scientific knowledge (Curran, 2009) across their respective areas. The second set of papers in this special issue raises awareness of new and emerging design, psychometric, and methodological issues and solutions to these issues.

This issue serves in part as a follow-up to the 2018 Prevention Science special issue, entitled “Who Benefits from Programs to Prevent Adolescent Depression?” (Howe et al., 2018), which had a primary focus on heterogeneity of intervention effects on depressive symptomatology in adolescence. The 2018 special issue led by Howe and colleagues included a series of papers detailing results from single randomized controlled trials (e.g., Connell et al., 2018; Garber et al., 2018; Mauricio et al., 2018), in addition to individual-level evidence syntheses (e.g., Brown et al., 2018), as well as a smaller section of methodologically oriented papers that focused on the combined issues of measurement, assessment timepoint, and sample heterogeneity in combined IDA/MIPDs (Brincks et al., 2018) and handling (and cautions) of missing data in data synthesis studies (Siddique et al., 2018). Together, that groundbreaking collection of papers had a tremendous impact on the field, as well as federal funding for work on IDA, MIPD, and related methodologies (Pearson & Sims, 2023; Reider & Sims, 2016), leveraging extant datasets addressing significant questions that have advanced prevention science. As such, it seemed timely for an update to that work, enabling us to both reflect on progress and chart a path forward for the field. In the sections that follow, we summarize the papers that were selected for publication, all of which underwent peer review. Also included in the special issue are two invited commentaries (Howe & Brown, this issue; Pearson & Sims, 2023), which provide additional insights and reflections on the collection of papers and highlight future directions for the field of prevention science.

Overview of the Papers Included in the Special Issue

This current issue presents a series of applied individual-level evidence synthesis studies across multiple areas including and beyond adolescent depression preventive interventions (e.g., brief alcohol interventions, suicidality, bullying) that leverage either IDA methodology, MIPD methodology, or both; in the case of MIPD, we include studies that are more “formalized” MIPDs that went through full systematic searches of the literature with a defined study universe for study selection (e.g., Mun et al., this issue; Schweer-Collins et al., 2023), whereas others did not (i.e., “pseudo-MIPDs”; Morgan-López et al., 2022a); however, they used the statistical methodology associated with MIPD analysis (e.g., Cooper & Patall, 2009). Further, new approaches and thought regarding (a) issues in logical and semantic harmonization, (b) transportability of IDA/MIPD findings to a target population, (c) handling variation in non-normal outcome distribution type in MIPDs, and (d) the idea of secondary analysis quasi-experiments as a novel form of IDA round out the latter section of this special issue. Several of these papers present both novel methodological issues and solutions for individual-level data synthesis and represent the current state of evidence synthesis for a given area of prevention (e.g., handling zero-inflation in brief alcohol intervention MIPDs; Mun et al., this issue).

Interestingly, in the intervening five years between the Howe et al. (2018) special issue and the current special issue, the state-of-the-art has progressed rapidly regarding multiple issues that are relevant to IDAs and/or MIPDs; this is reflected in the applied papers in the current collection of papers. For example, in the 2018 IDA studies, the estimation of commensurate scale scores across studies, and addressing cross-study (and other forms of) measurement non-invariance was largely conducted using an item response theory (IRT) approach. In contrast, many of the studies in the current special issue that have IDA components use moderated nonlinear factor analysis (Cole et al., 2022) which handles measurement noninvariance in a much more flexible manner than IRT but had only recently been introduced (Bauer, 2017) by time the 2018 special issue was published. Also, because individual-level evidence synthesis studies in prevention would have at least three levels of aggregation (i.e., repeated measures clustered within participants, participants clustered within studies), multilevel multiple imputation (MI) could be seen as a necessary tool for addressing missing data in IDA and/or MIPD studies. However, in 2018, multilevel MI was not readily available (but see Van Buuren & Groothuis-Oudshoorn, 2011), and there were many more warnings about the dangers of failing to model multilevel missingness within (Siddique et al., 2018) and outside (Gottfredson et al., 2017) of IDA/MIPD than there was accessible and user-friendly software to apply multilevel MI. Such software now exists in 2023 (e.g., Keller & Enders, 2021; Muthén & Muthén, 1998–2017) in ways that were not available as recently as five years ago.

Novel Findings for Cumulative Science Across Prevention

All of the studies under this subheading either (a) present new evidence in relation to individual-level evidence synthesis in a particular area of prevention and extend, in some cases, previous syntheses that used conventional summary data meta-analytic methods (e.g., Schweer-Collins et al., 2023) or (b) illuminate and/or separate the contextual effects of interventions from intervention effects on individual participants (e.g., Berry et al., 2022; Dong et al., 2022); simultaneous estimation of individual and contextual effects (e.g., school-level, study-level) have long been noted as one of the primary advantages of MIPDs over summary data MAs (Glass, 2000; Stewart & Parmar, 1993). Other studies used IDA and/or MIPD methodologies in order to explore questions regarding the so-called crossover effects (Reider & Sims, 2016), where interventions designed to target one or multiple proximal outcomes are assessed for their effects on distal outcomes that were not the original targets of intervention.

IDA/MIPD: Extension of Previous Evidence Syntheses

MIPD on Antibullying Interventions

Hensums et al. (2022) note that that anti-bullying interventions have generally shown robust effects in SDMAs, but (a) variability in effect sizes across different anti-bullying SDMAs and single trials appear to be tied to variation in programming components, yet component-level analyses require individual-level observations, and thus cannot be examined in SDMAs. In their MIPD, Hensums et al. (2022) found overall effect sizes in the small range, with no differences in effect sizes across intervention component type; they did find stronger effects of antibullying interventions at younger ages and for youth who had high rates of pre-intervention victimization; these latter findings likely could not have emerged without the benefits of considerable statistical power advantages for individual-level effects within an MIPD (Stewart & Parmar, 1993).

Moderator Effects in Incredible Years Program MIPD

Parent support interventions have shown positive effects on a variety of child outcomes. Nonetheless, the literature on whether parent support interventions benefit low-SES families differentially is not clear based on mixed findings from a meta-synthesis of systematic reviews and limited meta-analyses of SES moderation. Berry et al. (2022) extend an MIPD of the Incredible Years Program (Gardner et al., 2017) by detailing the moderating impact of SES; they found that low SES contributed to reductions in engagement in parenting, despite no differences in intervention efficacy for the Incredible Years Program, which may better explain the inconsistencies in the impact of SES and parent engagement on child outcomes found in other forms of evidence synthesis for parenting interventions.

Brief Alcohol Interventions: Comparing MIPD to SDMA

Brief interventions for alcohol relapse prevention have been shown to have mixed effectiveness in general. Even less clarity exists with regard to their efficacy within primary care settings based on both single RCTs and SDMAs. Brief intervention effectiveness appears to vary as a function of patient characteristics in SDMAs, but Schweer-Collins et al. (2023) correctly note that intervention moderators that operate at the individual-level may not be properly captured in study-level information (i.e., “ecological fallacy” Lubinski & Humphreys, 1996). Schweer-Collins et al. (2023) compare findings from their previous SDMA (Tanner-Smith et al., 2018) against an MIPD, based on a subset of individual-level data made available by investigators who contributed summary data to the SDMA. Schweer-Collins et al. (2023) found that women saw greater reductions in alcohol use and consequences from brief interventions in primary care, potentially due to higher rates of intervention-seeking by women after brief interventions. Importantly, this finding based on individual participant data was the direct opposite of findings from SDMAs that found men benefitted more. However, SDMAs can only look at moderators at the study-level (i.e., percent of men in the sample) rather than individual-level (i.e., the individual’s biological sex).

IDA/MIPD: Beyond Single Trial Results

Universal School-based Intervention MIPD and Disparities Reductions

A number of the IDA and/or MIPD studies presented in this special issue advance various aspects of prevention in areas where little-to-no formalized evidence synthesis previously existed. For example, the premise of the Dong et al. (2022) paper was predicated on the notion that parent engagement in children’s education has been known to predict educational, social, and behavioral outcomes prospectively, but both gender and racial/ethnic disparities in these outcomes grow over time. Our understanding of educational outcome disparities is mixed, likely due to low power within single trials to detect evidence of SES, gender, and race/ethnicity moderation effects for universal interventions. The Dong study, conducted under what we would consider the MIPD analytic framework (although they refer to it as IDA), is the first evidence synthesis that estimates cross-study effect sizes tied to behavioral outcome disparities reductions across evidence-based universal interventions (e.g., PBIS, GBG, incredible years).

Naturalistic Study of Interrelations Between Depression and Externalizing

Although interrelations between externalizing and depression have long been known, Magee et al. (2022) argue that the specificity in the “dynamic interplay of how emotional and behavioral vulnerabilities interact to increase risk for depression and externalizing problems” remains elusive and few longitudinal studies exist to address this (Kerr et al., 2012). Magee et al.’s (2022) paper addresses this by conducting an IDA of four datasets (three family checkup datasets and the Pittsburgh Girls Study) to examine interrelations between growth parameters for depression, externalizing, and inhibitory control. They provide greater clarity on downstream interrelations between depression, externalizing, and inhibitory control over a span from early to late adolescence. They also provide helpful insights from a measurement perspective in deattenuating these interrelations by using MNLFA (as opposed to total scores; Curran, 2009), where scale score estimation that accounts for the relative weights of items/symptoms serve to reduce measurement error (Kush et al., 2023) and measurement bias (Morgan-López et al., 2022b, 2023).

Peer Network Counseling Effects on Substance Use

Russell et al.’s (2022) review results from a series of randomized trials testing a peer network counseling intervention targeting adolescent and young adult substance use. The goal of peer network counseling was to reduce escalation of substance use by focusing on peer relationships as the primary context for initiating behavioral change. Russell et al. (2022) note that these trials have shown different effects of peer network counseling by gender (males in some studies, females in others) and substance use outcomes (alcohol versus cannabis). The authors used the MIPD framework to synthesize these trials for examination of these moderators, finding no moderation by gender or race/ethnicity, but finding moderation by baseline substance use levels.

Indirect Effects of Family Checkup on Youth Depression

Although it has been long-established that depression in parents is a risk factor for negative affectivity for youth, Seidman et al. (2022) argue that findings have been mixed with regard to whether parenting interventions such as the family checkup (FCU) achieve their impact on youth through reductions in parent depression (Saavedra et al., in press). Seidman et al. (2022) argue that part of the reason for the mixed findings is due in part to reduced power to detect mediation effects within any single trial. Seidman et al. (2022) found a significant intervention effect on maternal depression was observed across the three trials, with the FCU predicting improvements in maternal depression. In turn, such improvements predicted a reduction in the growth in both parent and youth reports of youth depressive symptoms across 10 years post baseline.

Its Your Game (IYG) Impact on Sexual Risk

The Its Your Game (IYG) intervention has previously shown moderation effects for reductions in risk for sexual initiation in middle schoolers using finite mixture modeling to assess classes of combinations of potential moderators. What was as yet unknown, and for which power would be negligible in any single trial, was whether IYG effects were moderated by classes of both individual- and school-level moderators. Vasilenko et al. (2022) used an MIPD analysis, but without necessarily defining and sampling studies from a formal universe of sexual risk-taking interventions, which showed that IYG was efficacious for individuals with low baseline risk and in schools with higher proportions of non-English speakers.

IDA/MIPD: Intervention “Crossover” Effects

Three studies in this special issue illustrate IDAs and/or MIPDs that highlight “crossover” effects. The Tiberio et al. (2023) study synthesizes data across multiple trials examining interventions for youth in foster care, including kindergarten-targeted youth in foster care for readiness for school (KITS), KEEP which targets dysregulated behaviors across development, MSS-Links for transitions to middle school, and TFCO which targets foster care youth who have already developed severe emotional and behavioral disorders. The interest was in examining potential moderating effects of biological sex, developmental period, number of foster care placements, and race/ethnicity. None of the singular trials would have had sufficient power to assess moderation across foster care interventions. The interventions reduced symptoms at the end of the interventions, but with sustained effects at follow-up assessments only for the most intensive form of intervention (TFCO) and with no evidence of moderation.

Connell et al. (2022) address whether the family checkup (FCU) intervention is efficacious in reducing long-term risk for suicidal behaviors, an effect that has recently been termed “suicide inoculation” (Morgan-López et al., 2022a) to distinguish reduction of risk for future suicide from “suicide prevention,” which has been associated more with the reduction of acute suicidal distress. A series of separate RCTs on family-focused prevention programs that were never intended to target suicide directly have shown reductions in suicide risk (Reider & Sims, 2016; Vidot et al., 2016). Consequently, NIMH has developed interest in IDA and/or MIPD studies of upstream interventions for suicide inoculation (Ayer et al., 2023; Pearson & Sims, 2023; Reider & Sims, 2016). In harmonizing data across trials, significant long-term effects of the FCU on reductions in suicide risk were observed, although differences between intervention and control group trajectories declined over time, with no moderation effects observed by gender or race/ethnicity.

It had been long-speculated that early treatment of anxiety disorders in children could have distal preventive effects on long-term substance use outcomes. Saavedra et al. (this issue) argued that many pediatric anxiety treatment trials were never designed to answer questions regarding long-term outcomes from a causal inference perspective. Many anxiety treatment RCTs have waitlist controls or evidence-based interventions as the comparator (Saavedra et al., 2010). In contrast to many prevention trials where a “no intervention” comparator is acceptable (e.g., school-as-usual in school-based trials), treatment cannot ethically be withheld from treatment-seekers. Thus, Saavedra et al. designed a novel form of IDA (secondary analysis quasi-experiment), integrating data from two child anxiety treatment trials of cognitive behavioral therapy (CBT) and a parallel psychiatric longitudinal epidemiological study of youth with anxiety disorders who were untreated. Using causal mediation methodology (MacKinnon et al., 2020), Saavedra et al. found that CBT had long-term effects on reduced risk for alcohol and substance use, but only to the extent that CBT led to reductions in anxiety by young adulthood. For “treatment-resistant” youth who were treated with CBT but whose anxiety did not remit, their substance use outcomes were worse than those who were untreated in childhood.

IDA/MIPD: Methodological Innovations in Prevention

Introduction to IDA

Zhao et al. (2022) present an overview and tutorial on the IDA process for readers who are new to IDA. They differentiate between types of evidence synthesis such as summary data meta-analyses and MIPDs and contrast these approaches against IDA. Zhao et al. (2022) then compare methods of estimating continuous latent variable scale scores and item parameters (e.g., item response theory [IRT], nonlinear factor analysis [NLFA]) and the relations between item parameters across the two frameworks. They give a brief overview of approaches to handling measurement noninvariance, also known as differential item functioning (DIF) such as multiple group confirmatory factor analysis (MG-CFA), multiple indicator/multiple cause (MIMIC) methods, and how moderated nonlinear factor analysis (MNLFA), used in many of the papers in this special issue, provides a flexible alternative for IDA. Zhao et al. (2022) also summarize the process of logical and semantic harmonization (e.g., deciding on harmonizable versus non-harmonizable item content across measures across studies) of item content across multiple measures of depression which varied across four studies. They also review the analytic harmonization process, covering testing for dimensionality, exclusion of items due to low base rates and/or small factor loadings, and DIF testing using a worked example harmonizing caregiver depression across multiple studies using SCL-90, PHQ-9, and WHOQoL. Saavedra and colleagues (this issue) also address this issue for specialized MNLFA models with low base rate/high item information items.

In-depth Considerations for Logical and Semantic Harmonization

Many innovations in IDA have primarily been made with regard to analytic harmonization methodologies for latent variables. While many of those same papers cover logical and/or semantic harmonization decisions that need to be made prior to analytic harmonization (see e.g., Hussong et al., 2013; Mun et al., 2016), many of these applications in practice can involve relatively simple decisions regarding semantic or logical harmonization of constructs across assessment systems because they either involved a very small and limited set of items for the construct of interest across studies (e.g., Curran et al., 2008) or diagnostic constructs based on item content derived directly from DSM criteria that were semantically equivalent across assessment systems (e.g., Morgan-López et al., 2022b). For example, McDaniel et al. (2023) present an in-depth examination of the logical and semantic harmonization process, with a focus on harmonization decisions for transdiagnostic constructs in prevention science for which no gold standards exist. The steps in the McDaniel et al. (2023) paper draw from parallels in test construction that are typical in the development of a new measure but apply and adapt these principles to a fixed item pool for IDAs with retrospective data. They detail steps of construct conceptualization and statement of purpose of the scale to be developed from the set of available items for the conceptualized constructs.

The IDA Measurement “Multiverse”

Cole et al. (2022) follow McDaniel et al. (2023) with an exploration the practical impact of a series of harmonization decisions in IDA—both logical/semantic and analytic—on the robustness of MNLFA factor score estimates, item parameters, and structural relations between delinquency and alcohol use across 72 different sets of possible harmonization decisions. They are interested in whether given the same sets of data for conducting an ILES study that has an IDA component would different sets of researchers who made different decisions at each step make the same inferences (a) regarding the measurement properties of the integrated set of measures and (b) regarding the substantive phenomenon of interest. In most cases, differences across combinations of IDA decision points did not appear to undercut (a) the consistency in scale score estimation across methods and (b) practical inferences on the relation between delinquency and alcohol use, though caution is warranted regarding the use of high factor score correlations (even as high as 0.98; McNeish, 2022) across methods as evidence of scale score synonymity, especially when there can be wide variation in scale score estimates under one model conditional on a specific score for scale scores estimated under a different model (Morgan-López et al., 2022c). The authors encourage applied IDA researchers to pursue alternate logical/semantic and analytic harmonization models to assess robustness of scale score estimates and substantive findings.

Novel Design Issues in Individual-level Data Synthesis

IDA and/or MIPD Methods for Low Base Rate Behaviors

One of the primary advantages to IDA and MIPDs that has been repeatedly touted over the years has been increasing the sample size for estimation of models with low base rate behaviors (Curran et al., 2008; Howe et al., 2018). However, low base rate behaviors can present challenges in the accuracy and precision of scale score estimation and outcomes modeling that have been underrecognized in the individual-level data synthesis space. Mun et al. (this issue) tackle this with regard to low base rate observed outcomes in the context of an MIPD of brief alcohol interventions that have been a part of the ongoing Project INTEGRATE study (Huh et al., 2019; Mun et al., 2016). Across the datasets that are a part of Project INTEGRATE, the authors specifically address zero-inflation (i.e., higher proportion of 0 s than expected from count distributions), overdispersion (i.e., violation of the distributional assumptions of the Poisson distribution), and cross-study variation in outcome distributions more broadly that is introduced when combining datasets with different inclusion criteria for baseline levels of alcohol use.

The authors address pros and cons regarding MIPDs where outcomes analysis of all datasets occurs simultaneously in a multilevel model where individual participants are clustered within studies (i.e., so-called 1-step MIPDs) or parameters are estimated separately for each study, then estimates and standard errors are combined in a dataset themselves, and a meta-regression model is then fit to the individual study results (“2-step” MIPD). The authors advocate for a 2-step MIPD approach in their specific case, largely because there is not (yet) an analysis model that is sufficiently general to estimate count outcomes with both overdispersion and excess 0 s, particularly across 3 levels of aggregation; either marginalized zero-inflated Poisson models (which lack of an overdispersion parameter) or negative binomial models (for studies with overdispersion, but where excess zeros are not problematic) were used, parameter estimates from each were saved, then meta-regression was conducted.

As noted above, Saavedra et al. (this issue) addressed the issue of low base rate latent variables by introducing zero-inflated MNLFA, a combination of MNLFA (Bauer, 2017) and a mixture IRT approach developed by Wall et al. (2015). The authors then compare graphically the distributional shapes of a series of scale score estimates to show how other approaches (e.g., conventional MNLFA) can distort the distributional shapes of low base rate latent variable scale scores in IDAs. Similarly, Musci et al. (2023) handle low-base rate psychosis symptoms by combining conventional MNLFA and mixture models with class constraints to assess trajectories of psychosis symptoms over development. By imposing a zero-class constraint in the general growth mixture model, Musci et al. (2023) were able to capture the heterogeneity that existed in the sample among those experiencing any psychosis during the analysis period. The demonstration of IDA methods combined with mixture modeling offers prevention researchers a novel way to assess heterogeneity in pooled studies.

Transportability Analysis in MIPDs

Barker et al. (2023) illustrate the complex interplay between intervention assignment, pre-intervention covariates, intervention outcomes, and trial membership in MIPDs in a framework called transportability analysis. While a number of the papers in this special issue present MIPD studies where each focal intervention is compared against each study-specific comparison condition as is often the case in conventional meta-analysis, Barker approaches the MIPD from the perspective of comparing each focal intervention condition against a common cross-study comparator condition, which is a form of individual-level network meta-analysis (Brincks et al., 2018; Dagne et al., 2016). This type of design, unlike conventional MA, combines similar intervention conditions across trials, which undercuts random assignment within each trial, essentially creating a multi-study quasi-experiment (Morgan-López et al., 2022a; Saavedra et al., 2021). For this, Barker et al. (2023) estimate propensity score weights that account for differences in the probability of intervention assignment across trials. What the Barker study also does simultaneous to modeling inverse probability of intervention assignment weights is attend to the issue of trial result generalizability, weighting results from trial results to account for differences in the relative propensity for participants to self-select into randomized control trials versus a larger population who may or may not likely participate in RCTs (Stuart et al., 2011).

A New Type of IDA: Secondary Analysis Quasi-experimental IDA

To date, IDAs have either been used to address cross-study measurement variation as part of studies that were primarily (a) cohort sequential or “accelerated longitudinal” observational studies (Bauer & Hussong, 2009; Curran et al., 2008) or, more recently in prevention, IPD meta-analytic in nature (e.g., Brown et al., 2018; Mun et al., 2016). The Saavedra study introduces a novel form of IDA: the IDA secondary analysis quasi-experiment. Calls have been long-standing for the development of secondary analysis quasi-experiments pairing clinical and psychiatric epidemiological samples both for (a) contexts where RCTs are unethical and (b) for questions regarding real-world effectiveness of interventions with epidemiological comparison participants, as opposed to control participants from RCTs (Diener et al., 2022; Zurovaca et al., 2021). Saavedra et al. proposed such a secondary analysis quasi-experiment, in the context of answering questions regarding long-term, indirect secondary preventive effects of cognitive behavioral therapy on long-term substance use outcomes. The authors also recognized that combining epi and clinical samples can have the same issues for measurement integration as other types of IDAs (Curran et al., 2008). In addition to the previously-mentioned measurement issues (e.g., latent variable zero-inflation), the Saavedra study combined propensity score weight estimation for addressing a lack of randomization of combined participants to CBT assignment, causal mediation methodology (MacKinnon et al., 2020) to address CBT by mediator interactions that can emerge as a function of not being able to randomized participants to different levels of the mediator, and (c) incorporation of post-treatment confounding (Rosenbaum, 1984), given that non-CBT participants were more likely to seek other forms of help-seeking than participants.

Conclusions and Continuing Development in Individual-level Data Synthesis

The special issue concludes with two commentaries. The first, by Drs. Jane Pearson and Belinda Sims, discusses the papers in the special issue in relation the perspective of National Institute of Health more generally and the National Institute of Mental Health in particular. The authors highlight the ways in which the studies in the special issue advance knowledge and methodology more broadly since Howe et al. (2018) and the ways in which this special issue focuses on priorities highlighted by NIMH at the time and since (Goldstein & Avenevoli, 2018). The authors summarize their perspectives on intervention crossover effects in IDA/MIPD, continuing development of approaches to moderation in IDA/MIPD, various methodological innovations, and coming attractions with regard to recent and forthcoming funding opportunities at NIMH in the IDA/MIPD space.

The second commentary comes from two experts who arguably introduced individual-level data synthesis to prevention science: Drs. George Howe and C. Hendricks Brown. In a manner consistent with how we envisioned this special issue, the authors draw a throughline from their 2018 special issue, their contributions (and others) to the area of individual-level data synthesis to prevention prior to the 2018 special issue, and innovations in the current special issue, in what is essentially a written history of IDA/MIPD in prevention science. Howe and Brown then provide perspectives on, but not limited to, retrospective psychometrics, comparisons of individual-level data synthesis with summary data meta-analyses, semantic harmonization, advantages of harmonizing studies representing different ranges of the outcomes of interest, and effect heterogeneity (a primary topic in the 2018 special issue), while also raising additional issues across each of these topics in ways that may not otherwise covered in the papers in the special issue.

Taken together, this collection of innovative papers is intended to both prompt reflection on how far the field has come in a relatively short period of time and inspire future lines of methodological and substantive work that advance these applications in prevention science. A silent “hero” of this work is the hundreds of datasets that have served to drive this line of research, not to mention the hundreds of thousands of participants involved in the original studies that were harmonized. Without the investment in those data collection efforts by federal agencies like NIH and the Department of Education, this IDA and harmonization work would not be possible. Perhaps only through these methodological innovations has the field of prevention science come into this “second season” and thus is now able to realize yet another benefit of those initial study investments.

Also relevant to this dialogue is the incredible level of coordination, collaboration, and transparency that is required to actually conduct an IDA study, in which investigators work together and combine several different data sets. Whereas prevention scientists have traditionally worked independently, in small teams, or with a single intervention, this work typically involve multiple investigators working across teams, projects, interventions, and study sites, openly sharing data and navigating many related challenges related to institutional review boards, data security, data sharing, authorship, etc. As the field moves toward embracing more team science and open science principles (Grant et al., 2022), and public data archiving becomes the norm (see NOT-OD-21–013), we feel strongly that these and other IDA and harmonization methodological innovations are especially timely and well suited for further realizing the return on investment in prevention science research.