Heterogeneity and Subgroup Analysis in Network Meta-Analysis

Abstract

Comprehensive healthcare decision-making requires a comparisons of the relevant competing treatment options for a particular disease state. Randomized controlled trials (RCTs) are considered the most credible evidence to obtain insight into the relative treatment effects of a medical intervention. However, an individual RCT rarely includes all competing interventions of interest. Typically, the evidence base consists of multiple RCTs where each of the available studies compares a subset of all the competing interventions of interest. If each of these trials has at least one intervention in common with another trial such that the evidence base can be represented with one connected network, a network meta-analysis (NMA) can provide relative treatment effects between all competing interventions of interest (see the network diagram in Fig. 18.1) (Ades 2003; Bucher et al. 1997; Dias et al. 2013a, 2018a; Hutton et al. 2015; Jansen et al. 2011, 2014; Lumley 2002; Lu and Ades 2004; Salanti et al. 2008). A NMA can be considered a generalization of conventional pairwise meta-analysis (Dias et al. 2018b, c). Rather than synthesizing the findings of multiple RCTs each comparing the same intervention with the same control, with a NMA we are simultaneously synthesizing the findings of multiple pair-wise comparisons across a range of interventions and obtaining estimates of relative treatment effects between all competing interventions based on direct and/or indirect evidence. Even if there was a conclusive RCT that included all competing interventions of interest, the available RCTs comparing a subset of the interventions provide relevant evidence as well. A NMA allows to estimate relative treatment effects based on the totality of the RCT evidence base.

1 Background

Comprehensive healthcare decision-making requires a comparisons of the relevant competing treatment options for a particular disease state. Randomized controlled trials (RCTs) are considered the most credible evidence to obtain insight into the relative treatment effects of a medical intervention. However, an individual RCT rarely includes all competing interventions of interest. Typically, the evidence base consists of multiple RCTs where each of the available studies compares a subset of all the competing interventions of interest. If each of these trials has at least one intervention in common with another trial such that the evidence base can be represented with one connected network, a network meta-analysis (NMA) can provide relative treatment effects between all competing interventions of interest (see the network diagram in Fig. 18.1) (Ades 2003; Bucher et al. 1997; Dias et al. 2013a, 2018a; Hutton et al. 2015; Jansen et al. 2011, 2014; Lumley 2002; Lu and Ades 2004; Salanti et al. 2008). A NMA can be considered a generalization of conventional pairwise meta-analysis (Dias et al. 2018b, c). Rather than synthesizing the findings of multiple RCTs each comparing the same intervention with the same control, with a NMA we are simultaneously synthesizing the findings of multiple pair-wise comparisons across a range of interventions and obtaining estimates of relative treatment effects between all competing interventions based on direct and/or indirect evidence. Even if there was a conclusive RCT that included all competing interventions of interest, the available RCTs comparing a subset of the interventions provide relevant evidence as well. A NMA allows to estimate relative treatment effects based on the totality of the RCT evidence base.

Fig. 18.1

Concept of network meta-analysis. An evidence network of connected randomized controlled trials for the competing interventions of interest (A, D, E, F, G, H) provide the data to estimate the relative treatment effects of each intervention relative to A (estimates for the basic parameters) given the assumption of consistency. These basic parameters are the basis for inferences regarding comparative effectiveness

RCTs of novel biopharmaceuticals are frequently performed in the context of regulatory approval. These trials are designed to demonstrate efficacy versus placebo or standard care, but not typically against each other. However, obtaining approval for drug licensing based on positive trials is not a guarantee for market access. Payers need to be convinced about the value of the new drug as well. As part of a health technology assessment (HTA), the value of the new intervention is assessed by examining its benefits, risks, and costs in comparison with existing standards of care for a given patient population of interest based on explicit and scientifically credible methods to inform healthcare and reimbursement decision-making. In many countries, the agencies tasked with HTA expect manufacturers of biopharmaceuticals to provide evidence regarding the comparative and cost-effectiveness of their drugs. With the evidence base characterized by multiple RCTs that only provide direct evidence regarding relative treatment effects for a subset of comparisons of interest, NMA is a core component of HTA submissions for new biopharmaceutical interventions.

It is well known that subgroups of patients within a population will derive value from a medical intervention that can differ systematically from the expected estimate of value for the overall population due to heterogeneous treatment effects (Espinoza et al. 2014; Sculpher 2008; Stevens and Normand 2004). As such, evaluation of treatment effects in subgroups are an integral part of the HTA review process, and treatment recommendations can be limited to specific subpopulations. In the context of relative treatment effects in RCTs, a subgroup effect can be understood as a categorical patient related covariate that modifies the treatment effect.

This chapter will discuss the estimation of relative treatment effects between competing interventions for specific subpopulations based on existing evidence by means of NMA methods. Methods that will be discussed include shrinkage estimation, network meta-regression, and a hierarchical approach to network meta-regression to combine study-level and patient level data.

2 Criteria for Valid Network Meta-Analysis

In order to appreciate the relevance of a NMA in the context of heterogenous and subgroup effects, it is important to highlight the criteria for a valid NMA first.

The purpose of a NMA is to estimate the relative treatment effects between competing interventions of interest for a specific target population based on available RCT evidence. In principle, this means that the study population in each of the RCTs that define the evidence base used for the NMA needs to be representative of the target population of interest. Individual RCTs are representative of the target population if there are no systematic differences in patient characteristics that influence the relative treatment effects, i.e. effect modifiers, between the study populations and the target population (Turner et al. 2009; Dias et al. 2018c). If this requirement for a relevant NMA is met, then there are no systematic differences in patient related effect-modifiers between the different RCTs in the network, and any of the relative treatment effects obtained with the NMA based on direct and/or indirect evidence is valid (Ades 2003; Dias et al. 2013a, 2018b, c; Jansen et al. 2012; Jansen and Naci 2013). Alternatively, if a subset of the trials in the network are not representative of the target population then there are differences in the distribution of effect-modifiers between the trials in the network and the estimated relative treatment effects based on indirect evidence are biased. If the study populations for all trials are different from the target population, but not different between trials, then the relative treatment effects of NMA are valid, but not representative of the target population of interest; we have external bias (Turner et al. 2009; Dias et al. 2018d). Obviously, effect-modifiers are not limited to patient characteristics. If there are study characteristics or contextual factors that act as effect modifiers and are different between the RCTs in the network, then the estimated relative treatment effects are biased as well.

In summary, for a credible NMA we need a connected network of RCTs where each trial has at least one intervention in common with another trial, without systematic differences in known and unknown effect modifiers between studies. For the findings to be relevant, there should not be systematic differences in effect-modifiers between the evidence base and the target population and setting of interest.

3 Standard Network Meta-Analysis Model

A NMA of RCTs relies on the same fundamental principle as a pairwise meta-analysis of RCTs. With a random effects pairwise meta-analysis, we assume that each study i aims to estimate study-specific relative treatment effects, δ _{i, AB}, and are exchangeable, i.e. a priori the study-specific relative treatment effects are expected to be similar, yet non-identical (Dias et al. 2013a, 2018b). The study specific treatment effects come from a normal distribution with mean d _AB and variance ${\sigma}_{AB}^2$ reflecting the between-study heterogeneity: ${\delta}_{i, AB}\sim N\left({d}_{AB},{\sigma}_{AB}^2\right)$. With a NMA we have multiple RCTs, each comparing a subset of all the interventions of interest, e.g. intervention A, B, C, and D (see Fig. 18.1). Now, we must assume the exchangeability of the study-specific relative treatment effects between any intervention k and b across the entire set of trials in the network: ${\delta}_{i, bk}\sim N\left({d}_{kb},{\sigma}_{kb}^2\right)$ (Dias et al. 2018b, c). We assume that the relative treatment effect δ _{i, AB} in trial i comparing B with A is a sample from the same random effects distribution as the other AB trials are estimating effects from, as well as the AC, AD, BC, and CD trials if these would have included intervention A and B as well. This notion extends to all the interventions in the network (Dias et al. 2018c). Accordingly, transitivity of the within-trial relative treatment effects of any intervention k relative to b in trial i can be described as δ _{i, bk} = δ _{i, Ak} − δ _{i, Ab}. Consequently, the average treatment effects are related according to: d _bk = d _Ak − d _Ab. Typically, it is assumed that ${\sigma}_{kb}^2={\sigma}_{Ak}^2={\sigma}_{Ab}^2={\sigma}^2$ (Dias et al. 2018b, c).

The general random-effects NMA model can be expressed as:

$$ g\left({\gamma}_{ik}\right)={\theta}_{ik}=\left\{\begin{array}{l@{\quad\quad\quad\qquad}l}{\mu}_i & k=b,\kern0.5em b\in \left\{A,B,C,..\right\}\\ {\mu}_i+{\delta}_{i, bk} & k\succ b \end{array}\right. $$

$${\delta}_{i, bk}\sim Normal\left({d}_{Ak}-{d}_{Ab},{\sigma}^2\right)\kern1em $$

(18.1)

where g is an appropriate link function (e.g. the logit link for binary outcomes) and θ _ik is the linear predictor of the expected outcome with intervention k in trial i (e.g. the log odds). μ _i is the study i specific outcome with comparator treatment b. δ _{i, bk} reflects the study specific relative treatment effects with intervention k relative to comparator b and are drawn from a normal distribution with the pooled relative treatment effect estimates expressed relative to the overall reference treatment A: d _bk = d _Ak − d _Ab (with d _AA = 0). Variance parameter σ ² reflects the heterogeneity across studies. With a fixed effects NMA, δ _ibk~Normal(d _Ak − d _Ab, σ ²) is replaced with δ _ibk = d _Ak − d _Ab because σ ² is assumed to be 0.

The primary parameters of interest are d _Ak and σ ² and are estimated based on the available RCTs included in the evidence network. Estimates of d _Ak reflect the relative treatment effect of each intervention k relative to overall treatment of reference A based on direct and/or indirect evidence and facilitates decision-making regarding how interventions rank regarding their effects on the outcome of interest. It is important to highlight that the model is applicable to many types of data, by just specifying an appropriate likelihood describing the data generating process and corresponding link function (Dias et al. 2018b). For example, when we have study level data and the measure of interest is response expressed as a proportion we use a binomial likelihood. When we have patient-level response (yes/no) data, we can use a Bernoulli distribution.

When the NMA is performed in a Bayesian framework, the parameters to be estimated, μ _i, d _Ak , and σ ² need be given prior distributions. In principle, we like the model parameter estimates to reflect the observed data from the RCTs and will therefore consider non-informative, or minimally informative prior distributions, wherever possible (Dias et al. 2018b). For example, μ _i~Normal(0, 100²), d _Ak~Normal(0, 100²), and μ~uniform(0, x) with x a reasonable upper bound dependent on the expected range of observed relative treatment effects.

4 Specific Challenges with Subpopulations

With a NMA we estimate relative treatment effects between competing interventions based on existing RCTs. The available trials for biopharmaceutical interventions have frequently been designed for regulatory approval and powered to detect a relative treatment effect for the overall study population. However, in the context of HTA, the target population of interest may be a subgroup of the overall study populations of the RCTs for the interventions of interest. This will pose challenges for the NMA when subgroup effects have not been reported or subgroup data are not available for the relevant trials. Even if the available RCTs do provide information on relative treatment effects for the subpopulation of interest, the studies may not have been powered to detect these subgroup effects and relative treatment effect estimates may be characterized by substantial uncertainty due to small sample sizes. In the following sections some of the methods that may be relevant for a NMA of treatment effects in subpopulations will be highlighted.

5 Shrinkage Estimation

Let us assume we have a connected network of RCTs that include all the competing interventions of interest, and all trials report results or provide data for mutually exclusive subgroups defined by observable patient characteristics. If the evidence base is rather weak due to a limited number of studies and/or small sample sizes in each of the subgroups, the NMA by subpopulation may not provide informative answers due to the uncertain estimates. As a potential solution we can consider using so called class effect models where the multiple interventions in the NMA are categorized into a smaller set of classes (Henderson et al. 2016; Kew et al. 2014; Lipsky et al. 2010; Mayo-Wilson et al. 2014; Warren et al. 2014). We assume that the treatment-specific relative effects within a class are exchangeable. For example, treatments with a similar mechanism of action fall into the same class and, a priori, their relative effects are more alike than effects of treatments from different classes (Dias et al. 2018d). For a class effects model with exchangeable treatment effects within a class, model 18.1 can be modified by defining that the basic parameters d _Ak are assumed to come from a distribution with a common mean and variance, if they belong to the same class:

$$ {d}_{Ak}\sim Normal\left({m}_{D_k},{\sigma}_{D_k}^2\right) $$

(18.2)

where D _k is defined as the class to which treatment k belongs. ${m}_{D_k}$ is the mean class effect in class D _k, and ${\sigma}_{D_k}^2$are the within-class variances. These models allow borrowing of strength across treatments in the same class: Unstable estimates for d _Ak due to limited subgroup data will be shrunken towards the class mean effect and become more precise than obtained with a model where d _Ak are assumed to be independent (Eq. 18.1). Depending on the sparseness of the available data, informative distributions may be needed for ${\sigma}_{D_k}^2$. It is recommended to perform multiple sensitivity analyses with different choices of values for the prior distributions of ${\sigma}_{D_k}^2$ (Dias et al. 2018d).

As an alternative to a NMA by subgroup, we can also define a model where all mutually exclusive subgroups are incorporated simultaneously and subgroup effects are exchangeable by treatment:

$$ g\left({\gamma}_{is,k}\right)={\theta}_{is,k}=\left\{\begin{array}{l@{\hspace*{5pc}}l}{\mu}_{is} & k=b,\kern0.5em b\in \left\{A,B,C,..\right\} \\ {\mu}_{is}+{\delta}_{is, bk} & k\succ b \end{array}\right. $$

$${\delta}_{is, bk}\sim Normal\left({d}_{s, Ak}-{d}_{s, Ab},{\sigma}_s^2\right) $$

$$ {d}_{s, Ak}\sim Normal\left({D}_{Ak},{\sigma}_k^2\right)\hspace*{147pt} $$

(18.3)

where θ _{is, k} is the linear predictor for the expected outcome with intervention k in subgroup s of trial i. μ _is is the expected outcome with comparator treatment b in subgroup s of study i. δ _{is, bk} reflects the relative treatment effect with intervention k relative to comparator b in subgroup s of trial i and are drawn from a normal distribution with the pooled estimates expressed in terms of the overall relative treatment effects versus treatment A in that subgroup d _{s, Ak}. This model adds the assumption that the underlying subgroup specific treatment effects d _{s, Ak} are drawn from a common normal distribution with mean D _Ak and treatment specific variance ${\sigma}_k^2$. With this model, highly uncertain relative treatment effects for each subgroup are stabilized by borrowing information from the data from other subgroups for that treatment (rather than from other treatments for the same subgroup) (Henderson et al. 2016).

The two “shrinkage models” presented here are a compromise between a model where treatment effects by subgroup are completely independent and a completely pooled analysis which ignores subgroup effects (Henderson et al. 2016).

6 Network Meta-Regression

Depending on the available data, a network meta-regression may be a relevant approach to estimate relative treatment effects between competing interventions for particular subpopulations. A meta-regression analysis can be used to explain between-study heterogeneity due to observed differences in the distribution of effect-modifiers between studies (Dias et al. 2018d). When there are differences between the target population of interest and the study populations of the individual studies included in the evidence network regarding effect-modifiers, a meta-regression can be used to adjust for this external bias (Turner et al. 2009; Dias et al. 2013b, 2018d). A network meta-regression analysis can be performed based on aggregate or study-level data, individual patient level data (IPD), or evidence networks where for a subset of the studies IPD is available and for other studies only aggregate level data (Efthimiou et al. 2016).

When the available evidence base only consists of aggregate level data, the model presented with Eq. 18.1 can be extended with covariates according to: (Cooper et al. 2009; Dias et al. 2013b, 2018d; Donegan et al. 2017, 2019; Nixon et al. 2007)

$$ {\theta}_{ik}=\left\{\begin{array}{l@{\qquad}l}{\mu}_{i} & k=b,\quad b\in \left\{A,B,C,..\right\} \\ {} {\mu}_i+{\delta}_{i, bk}+\left({\beta}_{Ak}-{\beta}_{Ab}\right)\left({m}_i-{x}_{target}\right) & k\succ b \end{array}\right. $$

$$ {\delta}_{i, bk}\sim Normal\left({d}_{Ak}-{d}_{Ab},{\sigma}^2\right)\hspace*{151pt} $$

(18.4)

m _i is the study-level covariate value for trial i, which can represent a subpopulation of interest. β _Ak represent the covariate effects with treatment k relative to the overall reference treatment A. x _target is the centered covariate value representing the target (sub) population of interest. d _Ak represent the relative effect of the treatment k compared to treatment A at the value x _target. As before, d _AA = β _Ak = 0. With this model we do not only assume consistency regarding relative treatment effects, but also regarding the parameters reflecting the impact of the covariates. Figure 18.2a, b illustrate the concept for an evidence base consisting of three AB and three AC trials for which only study-level data is available. If it is believed the Eq. 18.1 covariate of interest is not an effect modifier than the indirect BC estimate is relevant for the target population of interest (Fig. 18.2a). However, if the covariate is believed to be an effect modifier, a network meta-regression may be of interest to estimate the BC estimate for the target population (Fig. 18.2b). In model Eq. (18.4) the impact of the covariate on the relative treatment effects is assumed to be independent for each intervention k relative to A. However, we can also simplify the model by assuming the impact of the covariate is the same for every treatment k relative to A, β _Ak = B, or assume these to be exchangeable, ${\beta}_{Ak}\sim Normal\left(B,{\sigma}_B^2\right)$ (Cooper et al. 2009; Dias et al. 2013b, 2018d). This is useful when the number of studies for a certain intervention is limited or whether all the studies have the same covariate value.

Fig. 18.2

Network meta-analysis of AB and AC studies with a continuous covariate. (a) Indirect estimate for BC comparison without adjustment for the imbalance in the effect modifier; (b) with adjustment based on aggregate level data; and (c) after adjustment with individual patient level data (modified from Jansen 2012)

Without access to IPD we only have information on trial-level covariates. The information on patient characteristics is aggregated at the trial level, such as proportion of patients with prior treatment or severe disease, or mean age of the study population (Dias et al. 2013b, 2018d). If the study population of a particular trial is homogeneous regarding a certain dichotomous characteristic (e.g. only treatment naïve or treatment experienced), we have a dichotomous between-trial covariate. If the trial population is heterogeneous regarding a dichotomous characteristic (e.g. a mixed population of treatment naïve and experienced) the between-trial covariate is continuous representing the proportion of individuals with the characteristics in the trial. For an aggregated continuously distributed patient characteristic, the between-trial covariate is continuous as well. If the precision of each trial is large and the number of studies is small, we may find a spurious relationship based on the between-trial comparisons to be statistically significant if the contrast in the between-trial level covariate between these studies is sufficiently large (Dias et al. 2018d). On the other hand, with continuously distributed patient characteristics, the within-trial variation is typically much larger than the variation in aggregated means used for the between-trial meta-regression, thereby not having the power to detect a true relationship (Dias et al. 2013b, 2018d). Using aggregated information regarding patient characteristics in a network meta-regression is vulnerable to ecologic bias: the parameter estimate indicating the impact of a patient characteristic on a relative treatment effect based on between-trial comparisons may be very different from the within-trial relationship, as illustrated by the different regression lines in Fig. 18.2b, c (Berlin et al. 2002; Higgins and Thompson 2004; Jansen 2012; Lambert et al. 2002; Riley et al. 2010; Schmid et al. 2004; Riley and Steyerberg 2010).

Even in the absence of study level confounding, ecological bias can exist in non-linear models (Schmid et al. 2004; Greenland 2002; Jackson et al. 2006, 2008; Jansen 2012; Riley and Steyerberg 2010). In Fig. 18.3, the relationship between a relative treatment effect on the log odds ratio scale is presented against a dichotomous patient level effect-modifier X (x = 0 and x = 1) that is aggregated across the individuals in a study and represented as the proportion with x = 1. Let us assume that the probability of response is 40% with treatment A in both AB and AC trials for x = 0 and x = 1. The probability of response with B in the AB trials is 60% when x = 0 and 80% when x = 1. The probability of response with C in the AC trials is 70% when x = 0 and 95% when x = 1. The solid non-linear lines in Fig. 18.3a reflect the true log odds ratio of AB and AC trials for different distributions of the dichotomous covariate X in a particular study given the probability of the outcome with intervention A, B and C. For the AB comparison there are 5 studies in which the proportions of subjects with x = 1 are 0.1, 0.15, 0.20, 0.25, and 0.30 (the blue dots). For the AC comparison the five studies have proportions of x = 1 of 0.20, 0.30, 0.40, 0.50, 0.60 (the red dots). The dashed line in Fig. 18.3b reflects a network meta-regression model where the study specific log odds ratios are modeled as a function the proportion of subjects with x = 1 in each study using study-level data. When we are interested in the AB, AC, and indirect BC estimate for the subgroups x = 0 and x = 1, the projected treatment effects with the network meta-regression model are biased: the estimated log odd ratios for x = 0 and x = 1 (the dashed lines) differ from the true log odds ratios (the solid).

Fig. 18.3

Network meta-regression of AB and AC studies with a non-linear model with a dichotomous patient related covariate X to estimate subgroup effects x = 0 and x = 1 in the absence of study level confounding. (a) True indirect estimates for x = 0 and x = 1, can be obtained with patient level data for all studies; (b) Biased indirect estimates for x = 0 and x = 1 (b) based on aggregated level data network meta-regression (modified from Jansen 2012)

The limitations of network meta-regression based on aggregate level data can be overcome with the use of IPD. In the context of adjusting for external bias in relation to patient characteristics, a network meta-regression analysis based on IPD can be considered the “gold-standard” (Cope et al. 2012; Dias et al. 2018d; Debray et al. 2018; Leahy et al. 2018). The solid curves in Fig. 18.3a show the estimates that an analysis with IPD would provide (Jansen et al. 2012). With IPD available for all the trials in the evidence network, a model can be defined according to:

$$ {\theta}_{ij k}=\left\{\begin{array}{l@{\qquad\quad}l}{\mu}_i+{\beta}_{0i}{x}_{ij} & k=b,\quad b\in \left\{A,B,C,..\right\} \\ {}{\mu}_i+{\delta}_{i, bk}+{\beta}_{0i}{x}_{ij}+\left({\beta}_{Ak}-{\beta}_{Ab}\right){x}_{ij} & k\succ b\end{array}\right. $$

$$ {\delta}_{i, bk}\sim Normal\left({d}_{Ak}-{d}_{Ab},{\sigma}^2\right)\hspace*{150pt} $$

(18.5)

j reflects the individual in study i. β _0i is the main effect of covariate x on the outcome of interest in study i. x _ijis the value of the covariate for individual j in study i. Here we assume the interaction effect β _Ak is fixed across studies. We can also separate the within and between-trial interaction and define the model with a covariate for the mean value of the patient-related effect-modifier of each study and a covariate for the individual patient value of the effect-modifier minus the mean value in that study to describe the within-study variation (Riley and Steyerberg 2010; Donegan et al. 2013):

$$ {\theta}_{ij k}=\left\{\begin{array}{l@{\qquad}l}{\mu}_i+{\beta}_{0i}{x}_{ij}& k=b,\quad b\in \left\{A,B,C,..\right\}\\ {}{\mu}_i+{\delta}_{i, bk}+{\beta}_{0i}{x}_{ij}+\left({\beta}_{Ak}^a-{\beta}_{Ab}^a\right){m}_i & k\succ b \\ {}\quad +\left({\beta}_{Ak}^w-{\beta}_{Ab}^w\right)\left({x}_{ij}-{m}_i\right) & \end{array}\right. $$

$$ {\delta}_{i, bk}\sim Normal\left({d}_{Ak}-{d}_{Ab},{\sigma}^2\right)\hspace*{150pt} $$

(18.6)

${\beta}_{Ak}^a$ represent the between-study coefficient for the covariate effects with treatment k relative to the overall reference treatment A. ${\beta}_{Ak}^w$ represent the within-study coefficient for the covariate effects with treatment k relative to the overall reference treatment A. If the within-trial and between-trial interactions are different, then ecological or confounding bias may be present and the relative treatment effects may be biased. In such cases, inferences regarding treatment effects for specific subpopulations of interest should be based on the within-trial interactions estimated by models that separate the two types of interaction (Dias et al. 2018d). Again, models (18.5) and (18.6) can be simplified by assuming that the impact of the effect modifier is the same for every intervention k relative to A.

Unfortunately, a meta-analyst may not have access to IPD for all trials, but only for a subset. Rather than excluding relevant aggregate level studies from the network meta-regression, a combination of IPD studies and aggregate level studies is of interest (Sutton et al. 2008; Riley et al. 2007, 2008). We need a model that can combine both sources of data, such as (Donegan et al. 2013):

IPD trials:

$$ {\theta}_{ij k}=\left\{\begin{array}{l@{\quad}l}{\mu}_i+{\beta}_{0i}{x}_{ij}& k=b,\quad b\in \left\{A,B,C,..\right\} \\[3pt] {}{\mu}_i+{\delta}_{i, bk}+{\beta}_{0i}{x}_{ij}+\left({\beta}_{Ak}^a-{\beta}_{Ab}^a\right){m}_i & k\succ b \\[3pt] {}\quad +\left({\beta}_{Ak}^w-{\beta}_{Ab}^w\right)\left({x}_{ij}-{m}_i\right)& \end{array}\right. $$

Aggregate level data trials:

$$ {\theta}_{ik}=\left\{\begin{array}{l@{\qquad}l}{\mu}_{i} & k=b,\quad b\in \left\{A,B,C,..\right\} \\ {}{\mu}_i+{\delta}_{i, bk}+\left({\beta}_{Ak}^a-{\beta}_{Ab}^a\right){m}_i & k\succ b \end{array} \right. $$

$${\delta}_{i, bk}\sim Normal\left({d}_{Ak}-{d}_{Ab},{\sigma}^2\right) $$

(18.7)

With this model, the aggregate-level data studies as well as the IPD studies contribute to estimation of the between-study interactions. Now, let us assume we have a scenario where for a certain set of the direct comparisons in the network we have IPD and for the remaining set of direct comparisons we only have aggregate-level data. If treatment-by-covariate interactions are believed to be the same for each intervention k relative to A then depending on how IPD and study level data is divided over the available direct comparisons in the network, we may be able to “transfer” the within-trial interaction estimate for Ak comparison for which IPD is available to the Ak comparisons for which we only have aggregate level data. Unfortunately, this only applies to specific evidence structures. We can potentially improve the precision of the interaction effects for studies with only aggregate-level data for any structure based on the available IPD, if we define the model with the same interaction parameter for the within- and between-trial comparisons (Donegan et al. 2013; Jansen et al. 2012; Saramago et al. 2012). However, as mentioned, this will bias the estimates when there is study-level confounding, i.e. when not all effect-modifiers are accounted for, or when we have non-linear models (e.g. when adopting a logit link function).

7 Hierarchical Approach to Network Meta-Regression

As an alternative to using network meta-regression models that use the same interaction-effect parameters for the IPD studies and aggregate-level studies, we can use a type of model based on the hierarchical related regression approach to avoid aggregation or ecological bias (Jackson et al. 2006, 2008). A model for a hierarchical approach to a NMA (Jansen 2012; Phillippo et al. 2018) with a dichotomous covariate can be defined according to:

IPD trials:

$$ g\left({\gamma}_{ij k}\right)={\theta}_{ij k}=\left\{\begin{array}{l@{\qquad}l}{\mu}_i+{\beta}_0{x}_{ij} & k=b,\quad b\in \left\{A,B,C,..\right\} \\ {}{\mu}_i+{\delta}_{i, bk}+{\beta}_0{x}_{ij} &k\succ b \\\quad\ +\left({\beta}_{Ak}-{\beta}_{Ab}\right){x}_{ij} & \\\end{array}\right. $$

Aggregate level data trials:

$${\gamma}_{ik}={\gamma}_{ik}^o\left(1-{m}_i\right)+{\gamma}_{ik}^1{m}_i $$

$$ g\left({\gamma}_{ik}^o\right)={\theta}_{ik}^o=\left\{\begin{array}{l@{\qquad}l}{\mu}_{i} & \hspace*{85pt}k=b,\quad b\in \left\{A,B,C,..\right\} \\ {}{\mu}_i+{\delta}_{i, bk} & \hspace*{85pt}k\succ b \end{array} \right. $$

$$ \ g\left({\gamma}_{ik}^1\right)={\theta}_{ik}^1=\left\{\begin{array}{l@{\qquad}l}{\mu}_i+{\beta}_0 & k=b, \quad b\in \left\{A,B,C,..\right\} \\ {}{\mu}_i+{\delta}_{i, bk}+{\beta}_0+\left({\beta}_{Ak}-{\beta}_{Ab}\right) & k\succ b \end{array}\right. $$

$${\delta}_{i, bk}\sim Normal\left({d}_{Ak}-{d}_{Ab},{\sigma}^2\right) \hspace*{152pt} $$

(18.8)

The IPD part of this model is the same as the previous model (see Eq. 18.5), with the exception that the coefficient related to the covariate effect, β ₀, is fixed across studies. For the aggregate-level data part of the model, γ _ik is the expected outcome in study i with intervention k and is determined by integrating the individual-level model over the joint within-study distribution of the binary covariate. γ _ik equals the sum of the proportion of subjects with covariate x = 1 in each aggregate level data study (m _i) multiplied with ${\gamma}_{ik}^1$ and the proportion of subjects with covariate x = 0 (1-m _i) multiplied with ${\gamma}_{ik}^o$. ${\gamma}_{ik}^1$represent the marginal expected outcome with treatment k for a subject with the covariate x = 1 in study i. Similarly, ${\gamma}_{ik}^o$is the equivalent for a subject with x = 0. The essence of the approach is that an individual model is averaged over the population in study i to obtain the aggregate-level model for that study. Initial simulation studies have shown that relative treatment effects for subgroups with this approach may be less affected by bias than estimates obtained with network meta-regression for non-linear models with large treatment-by-patient-level-covariate interactions (Jansen 2012). Furthermore, it allows for the “transfer” of the within-trial interaction estimates to comparisons for which we only have aggregate level data available.

8 Conclusion

NMA provides a consistent framework to estimate relative treatment effects between competing interventions for a certain disease state based on RCT evidence. A NMA can be performed with study-level data, IPD, or a combination of both. In the context of health technology assessment, evaluation of treatment effects in subgroups are an integral part of the process to evaluate the value of a healthcare technology. Standard NMA to estimate subgroup effects may be challenging depending on the evidence available, and modified methods may be needed. If the evidence base is rather weak due to a limited number of RCTs and/or small sample sizes in each of the subgroups, the estimates obtained with a NMA may be stabilized with “shrinkage estimation” where intervention specific relative treatment effect are assumed exchangeable for interventions of the same class. As an alternative to a NMA by subgroup, we can also define a model where all mutually exclusive subgroups are incorporated simultaneously and subgroup effects are exchangeable by treatment, and thereby improving the precision of estimates. A network meta-regression may be a relevant approach to adjust for external bias when there are differences between the target population of interest and the study populations of the individual studies regarding effect-modifiers. When there is only access to study-level data, network meta-regression is vulnerable to ecologic bias when the subgroups of interest relate to patient characteristics. Even in the absence of study level confounding, ecological bias can exist in non-linear models. A NMA with IPD is the gold-standard to estimate relative treatment effects for different patient characteristics. Unfortunately, a meta-analyst may not have access to IPD for all trials, but only for a subset. Network meta-regression models can be defined where both sources of evidence are integrated. In order to improve the power to estimate interaction effects for comparisons for which only aggregate level data is available, we can use the same parameter for the within- and between-trial interaction effects at the expense of ecological or aggregation bias when there is study-level confounding or when we have non-linear models. A potential solution are hierarchical related regression models where the model for aggregate-level data is obtained by integrating an underlying IPD model over the joint within-study distribution of covariates. The methods presented in this chapter or only a selection of available techniques to perform evidence synthesis studies to estimate subgroup specific treatment effects. In principle, one could add comparative observational studies to the networks in an attempt to use a larger evidence base to estimate subgroup effects and effect-modification. However, additional research is needed regarding its benefit versus the potential risk of bias before any recommendations can be made. In general, the established NMA framework can be modified without violating its underlying assumptions of exchangeability and consistency in order to improve estimation of treatment effects for subpopulations based on the available RCT evidence. These methods are useful as part of drug development decisions and to support commercialization activities for novel drugs when preparing HTA submissions.