Some recent work on multivariate Gaussian Markov random fields

Abstract

Some recent work on conditional formulation of multivariate Gaussian Markov random fields is presented. The focus is on model constructions by compatible conditionals and coregionalization. Special attention is given to multivariate generalizations of univariate models. Beginning with univariate model constructions, a survey of key approaches to formulating multivariate extensions is presented. Two challenges in the formulation and implementation of multivariate models are highlighted: (1) entanglement of spatial and non-spatial components, and (2) enforcement for positivity condition. Managing the two challenges by decomposition, separation, and constrained parameterization is discussed. Also highlighted is the challenge of flexible modeling of (conditional) cross-spatial dependencies and, in particular, asymmetric cross-spatial dependencies. Interpretation of asymmetric cross-spatial dependencies is also discussed. A coregionalization framework which connects and unifies the various lines of model development is presented. The framework enables a systematic development of a broad range of models via linear and spatially varying coregionalization, respectively, with extensions to locally adaptive models. Formulation of multivariate models over variable-specific lattices is discussed. Selected models are illustrated with examples of Bayesian multivariate and spatiotemporal disease mapping. Potential applications of coregionalization models in imaging analysis, covariance modeling, dimension reduction, and latent variable analysis are briefly mentioned.

1 Introduction

Conditionally formulated Gaussian Markov random fields (GMRFs) and their multivariate generalizations have seen broad applications in image processing and analysis (Kashyap and Chellappa 1983; Mardia 1988), ecology (Lichstein et al. 2002), environmental and climate sciences (Daniels et al. 2006; Sain et al. 2011), mapping of gene frequencies (Gelfand and Vounatsou 2003), human brain mapping (Brezger et al. 2007), identification of genetic markers (Zhang et al. 2016), disease surveillance (Lawson 2013), and disease epidemiology and mapping (Besag et al. 1991; Kim et al. 2001; Jin et al. 2007), among others. The Markovian characterization of finite systems of spatially interacting random variables, which facilitates modeling spatial dependencies and enables local smoothing and borrowing information, is the primary motivation for the use of these models.

For example, in the context of Bayesian multivariate disease mapping, conditionally formulated multivariate GMRFs (MGMRFs), commonly known as multivariate conditional autoregressive (MCAR) models, or simply MCARs, have been developed to model spatial risk interactions within and across diseases (Kim et al. 2001; Jin et al. 2007). They are typically used for local risk prediction and borrowing information among related diseases or health outcomes, say, some are rare events while others are more common occurrences (Jin et al. 2007; MacNab 2009). MCARs have been formulated for spatiotemporal disease-risk smoothing in spatiotemporal disease mapping over geographic areas (Knorr-Held and Best 2000; MacNab and Gustafson 2007; Ugarte et al. 2017) and for analysis of multivariate spatiotemporal environmental pollutant monitoring data collected at fixed spatial locations and over time (Mardia and Goodall 1993; Daniels et al. 2006). They have been developed to model multivariate spatial dependencies among variables measured on regular grids (Sain et al. 2011). MGMRFs are also used as random effects or latent components priors in multivariate ecological regression models (Wakefield and Salway 2001; MacNab 2009), spatially varying coefficients models (Banerjee et al. 2014), time-to-event models (Carlin and Banerjee 2003), and spatial structural equation models (Liu et al. 2005; Congdon 2008a), to name a few.

Despite their wide-ranging applications, MGMRF models with simple cross-covariance functions, and separable models in particular, have been the most commonly used models in recent decades. The reasons for this are, perhaps, twofold. First, there have been limited choices of available models, such as non-separable models and those allowing for asymmetric cross-covariance functions. Second, the implementation of non-separable models is often computationally demanding and challenging, mainly due to the multi-dimensional nature of model constructions and the estimation and inference for constrained matrices of unknown parameters.

More recently, however, new and flexible MGMRF models have emerged. Efforts have been made in tackling computational challenges, notably in connection with hierarchical Bayesian analysis via Markov chain Monte Carlo (MCMC) simulation using WinBUGS software (Jin et al. 2007; MacNab and Gustafson 2007; Greco and Trivisano 2009; Martinez-Beneito 2013; Botella-Rocamora et al. 2015; MacNab 2016a, b). In particular, recent progress has been made in building flexible MGMRFs by constructing full conditionals (Sain and Cressie 2007; Sain et al. 2011) or by formulating latent full conditionals within a linear coregionalization framework (Jin et al. 2007; Greco and Trivisano 2009; MacNab 2016a, b).

This paper has three main objectives. The primary objective is to provide a survey of the key MCAR literature and a synthesis of the different lines of MCAR development, highlighting the challenges in formulation, implementation, and interpretation of conditionally formulated MGMRFs. The second objective is to present a broadened coregionalization framework for unified development of linear coregionalization and spatially varying coregionalization models, with extensions to locally adaptive models. The third objective is to discuss applications of MGMRFs in the context of Bayesian disease mapping, using illustrative examples of multivariate and spatiotemporal disease mapping. Potential applications of coregionalization MGMRFs in imaging analysis; covariance modeling; dimension reduction; and latent variable, component, and factor analysis are also briefly mentioned.

1.1 Some preliminaries

We will be concerned with the formulation of lattice models on finite lattices of integer-labeled n sites with a predefined system of local neighborhoods representing the neighbors of each site (Besag 1974; Besag et al. 1991). Let ${\varvec{W}}$ be a n by n connectivity matrix with elements $w_{ii} = 0$, $w_{ik} = 1$ if the ith and kth sites are neighbors (denoted $k \sim i$ hereafter) or $w_{ik} = 0$ otherwise. Let $w_{i+} = \sum _{k=1}^n w_{ik}$ denote the number of neighbors for the ith site. A lattice is named regular or irregular and “site” may refer to a point or region. An $n_r$ by $n_c$ grid is a regular lattice. In disease mapping, “sites” are often contiguous small geographical areas and they form an irregular lattice. The “neighborhood” for each of the areas is usually comprised of its adjacent neighbors, also named nearest-neighbors or first-order neighbors (see Besag 1974 for various lattice-neighbor systems).

We focus on the formulation of MCAR models for finite lattice systems of spatially interacting multivariate Gaussian random variates $\{ \zeta _{ij}, \forall i,j \}$, where $j=1,2,\ldots ,p$ is the labeling for the p variables. Arranging these np random variates in an n by p matrix $\varvec{\zeta }=\left[ \zeta _{ij} \right]$, and denoting the column vectors $\varvec{\zeta }_{\varvec{.}j} = (\zeta _{1j}, \ldots , \zeta _{nj})^{\top }$, $j=1,2,\ldots ,p$, and the row vectors $\varvec{\zeta }_{i \varvec{.}} = (\zeta _{i 1}, \ldots , \zeta _{i p})^{\top }$, $i=1,2,\ldots ,n$, they represent the variable and spatial domains, respectively. Denote $\hbox {vec}(\varvec{\zeta }^{\top }) = (\varvec{\zeta }_{1 \varvec{.}}^{\top },\ldots ,\varvec{\zeta }_{n \varvec{.}}^{\top })^{\top }$ and $\hbox {vec}(\varvec{\zeta }) =(\varvec{\zeta }_{\varvec{.}1}^{\top },\ldots ,\varvec{\zeta }_{\varvec{.}p}^{\top })^{\top }$. Without essential loss of generality, we will be concerned with zero-mean MGMRFs.

A conditionally formulated MGMRF has a fully specified precision matrix. Throughout the paper, MCAR constructions are discussed to yield joint precision matrices and expressed by $\hbox {vec}(\varvec{\zeta }^{\top }) \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varOmega }}_{\hbox {vec}(\varvec{\zeta }^{\top })})$ or, equivalently, $\hbox {vec}(\varvec{\zeta }) \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varOmega }}_{\hbox {vec}(\varvec{\zeta })})$, where $\varvec{{\varOmega }}_{\hbox {vec}(\varvec{\zeta }^{\top })}$ is a np by np block-matrix whose ikth block $\varvec{{\varOmega }}_{ik}$ is a square matrix of p-dimension and $\varvec{{\varOmega }}_{\hbox {vec}(\varvec{\zeta })}=\left[ \varvec{{\varOmega }}_{jl} \right]$ is a np by np block-matrix whose jlth block $\varvec{{\varOmega }}_{jl}$ is a square matrix of n-dimension. The matrix elements in $\varvec{{\varOmega }}_{\hbox {vec}(\varvec{\zeta }^{\top })}$ are sparse and spatially structured: The diagonal matrix elements of ${\varOmega }_{\hbox {vec}(\varvec{\zeta }^{\top })}$, $\varvec{{\varOmega }}_{ii}, \forall i$, characterize non-spatial conditional dependencies between variables at co-locations; the nonzero off-diagonal matrix elements $\varvec{{\varOmega }}_{ik} \ne {\varvec{0}}, k \sim i$, imply spatial and cross-spatial conditional dependencies. The covariance matrix $\varvec{{\varSigma }}_{\hbox {vec}(\varvec{\zeta }^{\top })}=\left[ \varvec{{\varSigma }}_{ik} \right] = \varvec{{\varOmega }}_{\hbox {vec}(\varvec{\zeta }^{\top })}^{-1}$ is, in general, a dense matrix: Its p by p block-matrix elements are covariance and cross-covariance matrix functions. These covariance and cross-covariance matrix functions do not, in general, have analytically transparent formulae. Throughout the paper, ${\varvec{U}}>0$ indicates that a matrix ${\varvec{U}}$ is positive definite.

1.2 Compatible conditionals: ideas, complexities, and challenges

We begin the discussion with univariate GMRFs as the simplest cases of MGMRFs. Consider a CAR model for n random variates $\varvec{\zeta }=(\zeta _1, \zeta _2, \ldots , \zeta _n)^{\top }$. At the heart of the CAR formulation is the characterization of Markovian dependence and independence through a set of compatible conditionals that imply conditional spatial dependence of $\zeta _i$ on its “neighbors” $\{\zeta _k, k \sim i \}$, $i=1,2,\ldots ,n$. These conditionals, also named full conditionals and denoted $f(\zeta _i|\{\zeta _k, k \sim i \})$, are site-wise local models. They are named compatible in the sense that, subject to highly restrictive consistency conditions (Besag 1974; Cressie 1993), together they define an unique GMRF whose precision matrix must be symmetric and positive definite. These are known as the symmetry and positivity conditions. In spatial statistics applications, and in disease mapping in particular, a common motivation for the use of a CAR model is to enable site-wise predictions through locally structured conditional auto-regressions, known as spatial smoothing.

GMRFs are formulated to imply neighbor-based conditional spatial autocorrelations and are also named spatial-interaction models (Besag 1974; Kashyap and Chellappa 1983). The lattice-neighborhood representation of spatially interacting random variates defines a conditional spatial autocorrelation structure over the Gaussian field, which often has two unknown parameters, denoted GMRF($c, \sigma$) hereafter, where c is a spatial interaction or dependence or smoothing parameter and $\sigma$ is a non-spatial scale parameter. Locally adaptive CARs that are parameterized by site-specific spatial parameters ${\varvec{c}}=(c_1,\ldots ,c_n)$ or site-specific scale parameters $\varvec{\sigma }=(\sigma _1,\ldots ,\sigma _n)$ have been explored for locally adaptive spatial smoothing (MacNab et al. 2006; Brewer and Nolan 2007; Reich and Hodges 2008) or for modeling locally structured heterogeneity (Congdon 2008b).

Compatible conditionals are formulated to give rise to MGMRFs that characterize multivariate spatial dependencies, including cross-dependencies between variables (Mardia 1988; Sain et al. 2011). There are two ways to formulating site-wise local models that give rise to p-variate GMRFs. We can treat each of the $\varvec{\zeta }_{i \varvec{.}} \hbox {s}$ as a p-variate local component and formulate MCARs using p-variate conditionals $f(\varvec{\zeta }_{i \varvec{.}}|\{\varvec{\zeta }_{k \varvec{.}}, k \sim i \}), i=1,2,\ldots ,n$ (Mardia 1988). Alternatively, we can consider each of the $\zeta _{ij}$ elements as a site- and variable-wise local component and formulate MCARs using univariate conditionals $f(\zeta _{ij}|\{\zeta _{kj}, k \sim i \}, \{\zeta _{kl}, k \sim i, l \ne j \}, \{\zeta _{il}, l \ne j \} ), i=1,2,\ldots ,n, j=1,2,\ldots ,p$ (Sain et al. 2011). A p-variate GMRF may be formulated to have a single spatial parameter c that influences spatial and cross-spatial dependencies or to have p variable-specific spatial parameters ${\varvec{c}}=(c_1,c_2,\ldots ,c_p)$ that allow for more flexible characterization of multivariate spatial dependencies (Gelfand and Vounatsou 2003; Carlin and Banerjee 2003; Jin et al. 2007; MacNab and Gustafson 2007; Martinez-Beneito 2013; Botella-Rocamora et al. 2015).

In recent MCAR literature, flexible p-variate CARs are usually parameterized by two p by p matrices ${\varvec{C}}$ and $\varvec{{\varSigma }}$ (or $\varvec{{\varGamma }}$) of spatial and non-spatial parameters, where $\varvec{{\varSigma }}>0$ (or $\varvec{{\varGamma }}>0$) is a non-spatial covariance (or precision) matrix that postulates non-spatial dependencies between variables at co-locations (Sain and Cressie 2007; Jin et al. 2007; Greco and Trivisano 2009; MacNab 2016a, b). These parameters are constrained to ensure that the MGMRF precision matrix is symmetric and positive definite. For example, constraints on MGMRF parameters may be discussed under the so-called diagonal dominance criterion, which guarantees the positivity condition if the MGMRF precision matrix is strictly diagonal dominant (Feingold and Varga 1962; Berman and Plemmons 1994; Sun et al. 1999; Rue and Held 2005; Sain and Cressie 2007; Greco and Trivisano 2009).

Univariate CARs have been built within a general framework developed in Besag (1974). Multivariate CARs, on the other hand, have been built in various ways. A general framework for formulation of multivariate GMRFs using multivariate conditionals is developed in Mardia (1988), with application to image processing. While separable models are readily derived within this framework (Mardia and Goodall 1993; Gelfand and Vounatsou 2003; Carlin and Banerjee 2004; Daniels et al. 2006; MacNab and Gustafson 2007), formulation and implementation of non-separable MGMRFs are less straightforward (Gelfand and Vounatsou 2003; Sain and Cressie 2007), primarily due to complex entanglement of spatial and non-spatial parameters, where enforcing the positivity condition may be computationally complex ( Sain and Cressie 2007, also see Sect. 3.3).

Alternatively, Sain et al. (2011) propose the formulation of MGMRFs using univariate conditionals. The proposal is a general framework for flexible modeling of multivariate spatial dependencies in terms of conditional spatial, cross-spatial, and site-wise non-spatial dependencies. The Sain et al. (2011) work was motivated and illustrated by a spatial analysis of multivariate output of regional climate computer models. The illustrative example concerned with modeling bivariate spatial dependencies of projected changes in temperature and precipitation on a spatial grid over the western USA. In the context of disease mapping, a similar bivariate CAR formulated using univariate conditionals, named a twofold CAR, was proposed by Kim et al. (2001).

In the present paper, we highlight two main challenges in formulation, implementation, and interpretation of MGMRFs by compatible conditionals: (1) the entanglement of spatial and non-spatial components, and (2) the enforcement for positivity condition. We show that even though the two challenges can be managed within the Mardia or Sain et al. framework in some situations, they become more manageable by formulating and implementing equivalent MGMRFs within a unified coregionalization framework presented herein. A common limitation of the two frameworks is that they do not readily facilitate non-separable MCAR generalization of all available univariate CARs. A notable example is the Leroux et al. CAR, where entanglement of its spatial and non-spatial parameters prevents a non-separable MCAR generalization within these frameworks. A solution to this limitation has recently been found within the linear coregionalization framework (MacNab 2016a, also see Sect. 4).

1.3 Coregionalization: a unified framework

In multivariate geostatistics, the so-called linear model of coregionalization is a popular approach for building valid covariance models and cross-covariance functions (Wackernagel 2003; Schmidt and Gelfand 2003; Gelfand et al. 2004; Zhang 2007; Genton and Kleiber 2015). The method concerns with representing a p-variate field as a linear combination of q independent univariate fields, usually $q < p$ for dimension reduction. Gelfand et al. (2004) develop non-stationary multivariate Gaussian process models through spatially varying coregionalization (SVC) for $q=p$.

In the context of Bayesian multivariate disease mapping, Jin et al. (2007) propose formulation of p-variate CARs by linear combination of p independent or correlated latent variables. They develop a linear coregionalization MCAR construction with multivariate latent components modeled by univariate conditionals that are parameterized by a symmetric matrix ${\varvec{C}}$ of spatial parameters. Greco and Trivisano (2009) extend the Jin et al. MpCAR to allow for an asymmetric matrix ${\varvec{C}}$ of spatial parameters. More recently, MacNab (2016a, b) presents a general linear coregionalization framework for the systematic development of coregionalization MCARs (cMCAR) as multivariate generalizations of univariate CARs. Linear combination of independent latent fields, also named the basic coregionalization framework, was shown to readily facilitate multivariate generalizations of any univariate CAR and CAR-related spatial models such as the Leroux et al. CAR (Leroux et al. 1999), the well-known Besag, York, and Mollie (BYM) model (Besag et al. 1991) or the modified BYM model (MacNab 2011).

A noteworthy benefit of formulating MCARs via linear coregionalization is that the spatial and non-spatial components representing coregionalization MGMRFs can be formulated separately, flexible modeling of spatial and cross-spatial dependencies among multivariate latent components is made possible (MacNab 2016a). Separate considerations of the spatial and non-spatial components also enable more flexible modeling of the non-spatial component in the coregionalization models. For example, order-dependent and order-free models may be formulated (MacNab 2016b). Linear coregionalization MCARs may enjoy some computational advantages (Martinez-Beneito 2013; Botella-Rocamora et al. 2015; MacNab 2016a, b), which will become more clear in the forthcoming discussion.

In the present paper, we connect linear coregionalization with the Sain et al. and Mardia frameworks and present a broadened coregionalization framework for flexible, inclusive and unified development of coregionalization MGMRFs via linear or spatially varying coregionalization, with extensions to locally adaptive models. We note that this extended framework opens new possibilities for alternative ways of formulating and implementing multivariate spatial models. We show that entanglement of spatial and non-spatial parameters is still present in the coregionalization models. However, the impact of the “entanglement” issue on the spatially varying coregionalization model characterization is notably different, potentially making these a more favorable class of covariance models in some applications.

1.4 Several related approaches to formulating MCARs

In multivariate disease mapping literature, multivariate extensions of univariate CARs are proposed under the shared component model framework (Knorr-Held and Best 2001; Held et al. 2005; MacNab 2010). This class of models may be formulated for multivariate spatial smoothing and for latent component analysis. We show in the present paper that a shared component model can have its linear coregionalization recast with a specific parameterization of the coregionalization coefficients matrix.

A MCAR can also be built as a product of a sequence of hierarchically formulated conditionals over the variable domain $\varvec{\zeta }=(\varvec{\zeta }_{\varvec{.}1}, \varvec{\zeta }_{\varvec{.}2}, \ldots , \varvec{\zeta }_{\varvec{.}p})$ (Royle and Berliner 1999):

$$\begin{aligned} f(\varvec{\zeta }_{\varvec{.}1}, \varvec{\zeta }_{\varvec{.}2}, \ldots , \varvec{\zeta }_{\varvec{.}p})=f(\varvec{\zeta }_{\varvec{.}1} | \varvec{\zeta }_{\varvec{.}2}, \ldots , \varvec{\zeta }_{\varvec{.}p}) f(\varvec{\zeta }_{\varvec{.}2} | \varvec{\zeta }_{\varvec{.}3}, \ldots , \varvec{\zeta }_{\varvec{.}p})\ldots f(\varvec{\zeta }_{\varvec{.}p}). \end{aligned}$$

For example, Jin et al. (2005) propose a hierarchically formulated multivariate CAR construction, named a generalized hierarchical MCAR or GMCAR. The GMCAR construction imposes a priori order among the variables, leading to order-dependent MCARs (Jin et al. 2007). MacNab (2016b) briefly discusses the connections between the Royle and Berliner (1999) approach and the linear coregionalization approach for MCAR construction.

Another approach to formulating MCARs is the Martinez-Beneito (2013) framework for building covariance models via matrix-variate theory. Working directly with the random variates matrix $\varvec{\zeta }= (\zeta _{ij})$ in the context of Bayesian multivariate disease mapping, Martinez-Beneito (2013) and Botella-Rocamora et al. (2015) propose MCAR models and discuss equivalents to Jin et al. (2007) MCARs. In the present paper, connections between the Martinez-Beneito (2013) framework and the coregionalization framework are discussed and established (in Sect. 5).

Carlin and Banerjee (2003) and Gelfand and Vounatsou (2003) propose MCARs built via decomposition of a CAR precision matrix, say, via Cholesky (Carlin and Banerjee 2003) or spectral (Gelfand and Vounatsou 2003) decomposition of a variable-specific proper CAR precision matrix. We further show in the present paper that the broadened coregionalization framework contains the Carlin and Banerjee and Gelfand and Vounatsou MCAR models as spatially varying coregionalization recasts.

1.5 Multivariate GMRFs as prior models and estimation of the prior parameters

MGMRFs are most commonly used as priors in Bayesian hierarchical models for modeling multivariate spatial or spatiotemporal lattice data (Cressie and Wikle 2011; Sain et al. 2011). In particular, univariate and multivariate GMRFs have been used as random effects priors in generalized linear mixed models (GLMM), with posterior inference implemented through MCMC simulations, say, using the Gibbs sampler. Some of the MCMC implementations have been coded in R (the R Development Core Team 2007) or C (Kim et al. 2001; Gelfand and Vounatsou 2003; Sain and Cressie 2007; Jin et al. 2007; Sain et al. 2011). A number of MGMRFs have been developed in the context of Bayesian multivariate or spatiotemporal disease mapping, with illustrative examples implemented in the freely available software WinBUGS (Spiegelhalter et al. 2007; MacNab and Gustafson 2007; Greco and Trivisano 2009; Martinez-Beneito 2013; Botella-Rocamora et al. 2015; MacNab 2016a, b).

Estimation of the spatial and non-spatial parameter matrices ${\varvec{C}}$ and $\varvec{{\varSigma }}$ in MGMRFs is a major challenge. Three solutions have been recently explored for Bayesian estimation of ${\varvec{C}}$. Jin et al. (2007) propose priors for the spectral decomposition of a symmetric matrix ${\varvec{C}}$. Greco and Trivisano (2009) and MacNab (2016a) propose priors for the singular value decomposition of an asymmetric matrix ${\varvec{C}}$. Both approaches ensure that all eigenvalues of the MGMRF precision matrix are strictly positive. Recognizing the possibility that the constrained reparameterization via spectral or singular value decomposition could put excessive a priori constraints on the elements of ${\varvec{C}}$, leading to shrinkage estimation of ${\varvec{C}}$ toward a diagonal matrix, MacNab (2016b) also propose alternative options of hierarchical priors (HPs) for the elements of a symmetric or an asymmetric matrix ${\varvec{C}}$. Bayesian estimation of the covariance matrix $\varvec{{\varSigma }}$ is usually carried out by placing priors on a variance-correlation decomposition of $\varvec{{\varSigma }}$ (Barnard et al. 2000; Jin et al. 2007; Martinez-Beneito 2013; MacNab 2016a, b) or on its symmetric square root factorization (MacNab 2016b), or by placing a Wishart prior on the precision matrix $\varvec{{\varGamma }}=\varvec{{\varSigma }}^{-1}$ (MacNab and Gustafson 2007).

In the CAR literature, the most well-known model is the improper intrinsic CAR (iCAR), which has a singular precision matrix of rank $n-1$ (Besag et al. 1991; Besag and Kooperberg 1995). In Bayesian disease mapping or ecological spatial regression analysis, for example, the iCAR model is commonly used as a random effects prior in GLMMs. Let $\varvec{\zeta }=(\zeta _1,\ldots ,\zeta _n)^{\top }$ be the iCAR modeled random effects in a GLMM. If unconstrained, the $\varvec{\zeta }$ confounds with the model’s intercept, leading to an identification problem. This well-known problem is often handled by placing a sum-to-zero consistency and identification constraint $\sum _{i=1}^{n}\zeta _i=0$ (Besag et al. 1991), or, more recently, by reparameterizing iCAR such that the resulting random effects are orthogonal to the model’s intercept (Reich et al. 2006; Hodges and Reich 2010; Goicoa et al. 2018). Goicoa et al. (2018) uncovered that a similar identification issue exists for the Leroux et al. CAR model (Leroux et al. 1999), whose precision matrix shares the same eigenvectors with the iCAR precision matrix but has different eigenvalues. They suggest a similar Leroux et al. CAR reparameterization for orthogonal random effects. In the present paper, we show that the iCAR and Leroux et al. CAR reparameterizations lead to covariance models that can be derived via coregionalization reconstructions. We also briefly mention analogous multivariate coregionalization reconstructions as multivariate priors for orthogonal random effects in multivariate spatial or spatiotemporal GLMMs.

1.6 Subsequent sections

Section 2 provides an overview of the Besag framework (Besag 1974). Two CAR constructions are presented; they subsume the major CAR models commonly seen in the literature.

Section 3 comprises surveys of building MCAR models by univariate conditionals and multivariate conditionals, respectively. The Sain et al. (2011) and Mardia (1988) MCAR frameworks are discussed for formulation of MGMRFs via (1) characterization of spatial, cross-spatial, and non-spatial conditional dependencies, and (2) decomposition of joint precision and covariance matrices. The latter is also discussed as a separation strategy that facilitates constrained parameterization for valid models and for statistical computation. The notions of separation and separability are discussed. The interpretations of MGMRF precision and covariance matrices, and, in particular, cross-spatial dependencies, are discussed.

Section 4 presents a coregionalization framework for building linear and spatially varying coregionalization models, with locally adaptive extensions. We also discuss coregionalization recasting and reconstructions of CAR models and priors for orthogonal random effects. The formulation of MGMRFs over component-specific lattice-neighbor schemes, as well as over different lattices, is discussed. The Martinez-Beneito framework is briefly outlined in Sect. 5, which highlights the connections between the Martinez-Beneito framework and the coregionalization framework.

Estimation of constrained spatial and non-spatial parameters in MGMRFs is briefly reviewed in Sect. 6. Multivariate GMRFs as prior models in Bayesian hierarchical modeling are discussed in Sect. 7. Bayesian computation via MCMC simulation is briefly discussed. Illustrative examples of Bayesian multivariate and spatiotemporal disease mapping are presented in Sect. 8. We end the paper with a summary discussion in Sect. 9.

2 Univariate CARs

A general CAR framework for a lattice-neighborhood system of zero-mean random variates $\varvec{\zeta }=(\zeta _1, \zeta _2, \ldots , \zeta _n)^{\top }$ is defined by a set of site-specific univariate auto-Gaussian conditionals $f(\zeta _i|\varvec{\zeta }_{-i})=f(\zeta _i| \{\zeta _{k}: k \sim i \})$ (Besag 1974):

$$\begin{aligned} \hbox {E}(\zeta _i|\varvec{\zeta }_{-i}) = \sum _{k=1}^n \beta _{ik}\zeta _{k} w_{ik}= \sum _{k \sim i} \beta _{ik} \zeta _{k}, ~~ \hbox {Prec}(\zeta _i|\varvec{\zeta }_{-i}) = \tau _i, ~~i=1,\ldots ,n, \end{aligned}$$

(1)

which yield a family of GMRFs with the following joint precision and covariance matrices:

$$\begin{aligned} \varvec{{\varOmega }}_{\varvec{\zeta }}= \varvec{\tau }({\varvec{I}}_n - \varvec{\beta }), ~~\varvec{{\varSigma }}_{\varvec{\zeta }}= ({\varvec{I}}_n - \varvec{\beta }) ^{-1} \varvec{\sigma }^{2} , \end{aligned}$$

(2)

provided that $\varvec{\tau }\varvec{\beta }= \varvec{\beta }^{\top } \varvec{\tau }$ (for the symmetry condition) and $\varvec{\tau }({\varvec{I}}_n - \varvec{\beta }) > 0$ (for the positivity condition), where $\varvec{\tau }= \hbox {diag}(\tau _1, \ldots , \tau _n)$, $\tau _i =\sigma _i ^{-2}$, $\varvec{\sigma }= \hbox {diag}(\sigma _1, \ldots , \sigma _n)$, $\varvec{\beta }= \left[ \beta _{ik} \right]$, $\beta _{ii}=0$, $\beta _{ik} \ne 0, k \sim i$ and $\beta _{ik} = 0$ otherwise.

Several important facts implied by the CAR framework should be noted. The conditional auto-regressions in (1) postulate conditional spatial dependence of $\zeta _i$ on $\{\zeta _{k}: k \sim i \}$, for $i=1,2,\ldots ,n$. The CAR coefficients, a set of nonzero $\beta$-coefficients $\{\beta _k, k \sim i \}$ that characterize and control for conditional spatial dependencies, are often functions of spatial dependence parameter(s). The site-specific conditional variance $\sigma ^2_i$ is the variance of predicting $\zeta _i$ given $\{\zeta _{k}: k \sim i \}$, for $i=1,2,\ldots ,n$. In other words, the matrix $\varvec{\beta }$ (or $I_n-\varvec{\beta })$) and $\varvec{\tau }$ characterize the GMRF spatial and non-spatial components, respectively. Entanglement of spatial and non-spatial components can be present even in univariate GMRFs. Nevertheless, compatible conditionals are typically formulated to enable a variance-correlation decomposition (Barnard 2000) and separation of the spatial and non-spatial components with respect to the joint precision matrix, as expressed by (2) (discussed in Sect. 3.2).

For example, let $\sigma _i^2= \sigma ^{2} m_i^{-1}$ and $\beta _{ik}=c m_i^{-1} w_{ik}$ in (1), where the $m_i \hbox {s}$ are known scaling factors. One can derive a CAR construction yielding precision matrix

$$\begin{aligned} \varvec{{\varOmega }}_{\varvec{\zeta }}(c, \sigma )= \sigma ^{-2} ({\varvec{D}}_m - c {\varvec{W}}), \end{aligned}$$

(3)

where ${\varvec{D}}_m=\hbox {diag}(m_{1}, m_{2}, \ldots ,m_{n})$, provided that $({\varvec{D}}_m - c {\varvec{W}}) > 0$. Expression (3) represents a variance-correlation decomposition and the separation of spatial and non-spatial parameters such that the positivity condition is guaranteed by placing a constraint on c, independent of $\sigma$:

$$\begin{aligned} c \in (c_{\tiny \hbox {min}}, c_{\tiny \hbox {max}}), \end{aligned}$$

(4)

where $c_{\tiny \hbox {min}}$ and $c_{\tiny \hbox {max}}$ are the reciprocals of the minimum and maximum eigenvalues of ${\tilde{{\varvec{W}}}} = {\varvec{D}}_m^{-1/2} {\varvec{W}}{\varvec{D}}_m^{-1/2}$ (Sun et al. 1999). Constraint (4) is a sufficient and necessary condition to ensure that the eigenvalues of $({\varvec{D}}_m - c {\varvec{W}})$ are strictly positive, which is a sufficient and necessary condition for $({\varvec{D}}_m - c {\varvec{W}}) > 0$ (Sun et al. 1999).

This construction contains three well-known CARs commonly seen in the literature: (1) the basic CAR or bCAR, when $m_i=1$, (2) the proper CAR or pCAR, when $m_i=w_{i+}$, and (3) the previously mentioned intrinsic CAR or iCAR, when $c=1$ and $m_i=w_{i+}$. The bCAR is commonly used to model data on a regular grid (Oliveira 2012; Sain et al. 2011). The latter two are popular for hierarchical modeling of spatial data on irregular lattices, say, in the context of small area disease mapping (see Lawson 2013 for a recent review). A discussion of iCAR reparameterization, as previously mentioned, is given in Sect. 4.4.

Consider another CAR construction by letting $\sigma _i^2 = \sigma ^{2} (1-c+c m_{i})^{-1}$ and $\beta _{ik} = c ~ (1-c+c m_{i})^{-1} w_{ik}$ in (1), which leads to a joint precision matrix:

$$\begin{aligned} \varvec{{\varOmega }}_{\varvec{\zeta }}(c, \sigma ) = \sigma ^{-2} \left[ c ({\varvec{D}}_m - {\varvec{W}})+(1-c)~{\varvec{I}}_n \right] , ~ c \in (0,1). \end{aligned}$$

(5)

In the present paper, (5) is named the LCAR construction: It contains the Leroux et al. CAR when $m_i=w_{i+}$ (Leroux et al. 1999). Entanglement of the spatial and non-spatial components is readily seen from the LCAR conditionals and from expression (5): The site-specific conditional variances are functions of both the spatial and scale parameters. The Leroux et al. CAR precision matrix is a weighted sum of the iCAR precision matrix and a precision matrix of a Gaussian field of IID components.

The constructions (3) and (5) subsume the CAR models commonly discussed in the literature (Lee 2011; MacNab 2011, 2016a). In disease mapping, for example, the spatial parameter in a CAR (3) or (5) model is positive or assumed positive, which leads to positive conditional spatial autocorrelations that imply a tendency of spatial (neighborhood) similarity over space. The $\beta$-coefficients in the CAR (3) construction, say, the bCAR or pCAR model, can be both positive and negative (Sun et al. 1999; MacNab 2011, 2016a). However, the pCAR model is often used as a spatial smoother; this is done by containing $c \in (0,1)$ to assume positive conditional autoregressive correlations. These CAR models may have limited scope and flexibility to model complex spatial dependencies and spatial heterogeneities. Indeed, the literature on CAR models explores adaptive iCAR models that allow for site-specific conditional precision functions (Brewer and Nolan 2007; Reich and Hodges 2008) or adaptive LCAR models with site-specific spatial parameters ${\varvec{c}}=(c_1,\ldots , c_n)$ (MacNab et al. 2006; Congdon 2008b).

For illustrative purpose, we consider two adaptive CAR constructions in the present paper. Let $\tau _i^2 = m_i \sigma _i^{-2}$ and $\beta _{ik} = c_i ^{1/2} c_k^{1/2} ~ m_i^{-1} \sigma _i \sigma _k^{-1} w_{ik}$ in (1), we consider an adaptive generalization of the CAR (3) formulation, which gives rise to a GMRF construction with the following precision matrix:

$$\begin{aligned} \varvec{{\varOmega }}_{\varvec{\zeta }}^{\tiny \hbox {adaptive}} ({\varvec{c}}, \varvec{\sigma }) = \hbox {diag}(\varvec{\sigma }^{-1}) ({\varvec{D}}_m - \hbox {diag}({\varvec{c}})^{1/2} {\varvec{W}}\hbox {diag}({\varvec{c}})^{1/2}) \hbox {diag}(\varvec{\sigma }^{-1}), \end{aligned}$$

(6)

provided that $c_j \in (0, c_{\tiny \hbox {max}})$, ${\varvec{c}}=(c_1,\ldots ,c_n)$, $\varvec{\sigma }=(\sigma _1,\ldots ,\sigma _n)$. Notice that the $\varvec{\beta }$-coefficients in the adaptive CAR conditionals are functions of spatial and scale parameters.

In addition, we consider an adaptive generalization of the construction (5) by letting, in (1), $\tau _i = (1-c_i+c_i m_i) \sigma ^{-2}$ and $\beta _{ik} = c_i ^{1/2}c_k^{1/2} (1-c_i+c_i m_i)^{-1} \sigma _i \sigma _k^{-1} w_{ik}$, which leads to the following joint precision matrix:

$$\begin{aligned} \varvec{{\varOmega }}_{\varvec{\zeta }}^{\tiny \hbox {adaptive}} ({\varvec{c}}, \varvec{\sigma }) = \varvec{\sigma }^{-1} \left[ \hbox {diag}({\varvec{c}})^{1/2}(D_m-{\varvec{W}}) \hbox {diag}({\varvec{c}})^{1/2} + {\varvec{I}}_n-\hbox {diag}({\varvec{c}}) \right] \varvec{\sigma }^{-1}. \end{aligned}$$

(7)

The adaptive constructions (6) and (7) differ from the previously mentioned adaptive iCAR and LCAR models in two noteworthy ways. First, they are adaptive CAR constructions that reduce to their simpler adaptive or non-adaptive counterparts when $c_i=c$ and/or $\sigma _i=\sigma$. In addition, multivariate generalizations of these adaptive CAR constructions can be readily formulated (see Sects. 3 and 4 for examples). These adaptive models, perhaps with explanatory covariates, may allow for more flexible characterization of site-varying spatial interactions or more flexible spatial smoothing. These adaptive constructions may be particularly useful when explanatory covariates are available to model the site-varying spatial and/or scale parameters, say, in the context of joint mean and covariance (or inverse covariance) modeling in the spirit of the mean-covariance modeling discussed in the literature of mixed effects models for longitudinal data (e.g., Pourahmadi 1999; Pan and MacKenzie 2003, 2007). Congdon (2008b) provides an example where explanatory covariates are used to model the site-varying spatial parameters in a locally adaptive LCAR model.

3 Multivariate CARs defined by compatible conditionals

3.1 Multivariate CARs defined by univariate conditionals

The Sain et al. (2011) framework defines MCARs by univariate conditionals:

$$\begin{aligned} \hbox {E}(\zeta _{ij}|\varvec{\zeta }_{-ij}) = \sum _{k \sim i} \beta _{ijkj} \zeta _{kj} + \sum _{k \sim i,~ l \ne j} \beta _{ijkl} \zeta _{kl} + \sum _{l \ne j} \beta _{ijil} \zeta _{il},~~ \hbox {Prec}(\zeta _{ij}|\varvec{\zeta }_{-ij})= \sigma _{ij}^{-2},\nonumber \\ \end{aligned}$$

(8)

which, under regularity conditions, lead to a family of MGMRFs with a precision matrix $\varvec{{\varOmega }}= \varvec{\sigma }^{-2}~ ({\varvec{I}}_{np}-\hbox {Block}(\varvec{\beta }_{ik}))$, where $\varvec{\beta }_{ik} \hbox {s}$ are $p \times p$ matrices, $\varvec{\sigma }= \hbox {diag}(\sigma _{11}, \ldots , \sigma _{1p}, \ldots , \sigma _{n1}, \ldots , \sigma _{np})$. The Sain et al. framework is a direct extension of the Besag CAR framework. The conditional auto-regression $\beta$-coefficients postulate conditional spatial and cross-spatial dependencies in the first and second summations, respectively, and conditional dependencies between variables at co-locations in the third summation (Sain et al. 2011). The conditional variance $\sigma _{ij}^{2}$ is the variance of predicting $\zeta _{ij}$ given $(\{\zeta _{kl}, k \ne i , l \ne j \} )$ for $i=1,2,\ldots ,n, j=1,2,\ldots ,p$. The Sain et al. family of MCARs is denoted $\hbox {MCAR}_{\tiny \hbox {Sain et al. UC}}$ hereafter, where “UC” stands for univariate conditionals.

Following the Sain et al. (2011) proposal, a multivariate generalization of the univariate CAR construction (3) can be formulated by letting, in (8), $\sigma _{ij}^2 = m_{i}^{-1} \sigma _{j}^2$, $\beta _{ijil} = \rho _{jl}^c~\sigma _{j} \sigma _{l}^{-1} m_{i}^{-1}$, $\beta _{ijkl} = c_{jl}~\sigma _{j} \sigma _{l}^{-1} m_{i}^{-1} w_{ik} ~~ k > i, ~ \beta _{ijkl} = c_{lj}~\sigma _{j} \sigma _{l}^{-1} m_{i}^{-1} w_{ik} ~~ k<i$, which give rise to a MGMRF construction with the following precision matrix:

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}^{\tiny \hbox {MCAR}_{\tiny \hbox {Sain et al. UC}}}({\varvec{C}}, \varvec{\rho }^c, \varvec{\sigma }) = ({\varvec{I}}_n \otimes \varvec{\sigma }^{-1}) {\varvec{S}}(\varvec{\rho }^c, {\varvec{C}}) ({\varvec{I}}_n \otimes \varvec{\sigma }^{-1}), \end{aligned}$$

(9)

provided that ${\varvec{S}}(\varvec{\rho }^c, {\varvec{C}}) > 0$, where

• a) ${\varvec{C}}$ is the p by p matrix of spatial parameters mentioned earlier, ${\varvec{S}}(\varvec{\rho }^c, {\varvec{C}}) = ({\varvec{D}}_m \otimes {\varvec{I}}_{p} - {\varvec{I}}_n \otimes \varvec{\rho }^c - ({\varvec{W}}_U \otimes {\varvec{C}}+{\varvec{W}}_U^{\top } \otimes {\varvec{C}}^{\top }))$, ${\varvec{W}}_U$ is the upper triangular part of ${\varvec{W}}$ (Greco and Trivisano 2009), and

  $$\begin{aligned} {\varvec{W}}_U \otimes {\varvec{C}}+ {\varvec{W}}_U^{\top } \otimes {\varvec{C}}^{\top } = \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} {\varvec{0}}&{} {\varvec{C}}w_{12} &{} &{} \cdot \cdot \cdot &{} {\varvec{C}}w_{1n} \\ {\varvec{C}}^{\top } w_{21} &{} {\varvec{0}}&{} &{} \cdot \cdot \cdot &{} {\varvec{C}}w_{2n} \\ &{} \cdot &{} &{} &{} \\ &{} &{} \cdot &{} &{} \\ &{} &{} &{} \cdot &{} \\ {\varvec{C}}^{\top } w_{n1} &{} {\varvec{C}}^{\top } w_{n2} &{} &{} \cdot \cdot \cdot &{} {\varvec{0}}\\ \end{array} \right] , \end{aligned}$$

• b) $\varvec{\sigma }= \hbox {diag}(\sigma _1, \ldots , \sigma _{p})$ are scale parameters, and,

• c) $\varvec{\rho }^c =\left[ \rho _{jl}^c \right]$ is a p by p matrix with elements $\rho _{jj}^c=0$, $\rho _{jl}^c \in (-1,1), j \ne l$, $\rho _{jl}^c=\rho _{lj}^c$, they are the co-located between-variable partial correlation parameters.

By letting $m_i=1$ in (9), one can derive the Sain et al. (2011) MCAR model, which is a multivariate generalization of the bCAR (MacNab 2016a) and is denoted $\hbox {MbCAR}_{\tiny \hbox {Sain et al. UC}}$ hereafter. In addition, in (8) the conditional auto-regressions have a total of $p*w_{i+}+p-1$ terms in the three summations. Let $m_i = p*w_{i+}+p-1$ in (9) we name the model p-fold CAR, also denoted $\hbox {MCAR}_{\tiny p\hbox {-fold}}$ hereafter, for it represents a multivariate generalization of the twofold CAR proposed in Kim et al. (2001).

One can readily derive locally adaptive extensions of the MCAR construction (9) by allowing for site-varying variable-specific scale parameters $\sigma _{ij} \hbox {s}$, perhaps with different choices of scaling factors $m_{ij} \hbox {s}$. Specific MCARs may be built by placing a structural assumption on the partial correlation matrix $\varvec{\rho }^c$, for example, in a spatiotemporal model with the time dimension modeled by an autoregressive or random walk process (see Sect. 8 for illustrative examples). Furthermore, this framework enables a broader conceptualization of MGMRFs defined over different lattices (Sain et al. 2011, to be discussed in Sect. 4.5).

It should be noted that $c_{jl} \ne c_{lj}$ implies asymmetry of conditional cross-spatial dependencies in relation to the two variables and to the labeling of the neighborhood sites, say, those in the lower and upper triangular part of the connectivity matrix ${\varvec{W}}$, respectively. To further explain, and without loss of generality, let us consider the bivariate bCAR, a model discussed in Sain et al. (2011). The conditional spatial and cross-spatial dependencies characterized in the model may be illustrated via the following conditional expectations:

$$\begin{aligned} \hbox {E}(\zeta _{i1}|\varvec{\zeta }_{-i1}) = \sum _{k \sim i} c_{11} \zeta _{k1} + \sum _{k \sim i, \, k <i} c_{12} \left( { \sigma _1 \over \sigma _2} \right) \zeta _{k2} + \sum _{k \sim i, ~k >i} c_{21} \left( { \sigma _2 \over \sigma _1} \right) \zeta _{k2} + \rho ^c_{12} \left( { \sigma _1 \over \sigma _2} \right) \zeta _{i2}, \nonumber \\ \end{aligned}$$

(10)

$$\begin{aligned} \hbox {E}(\zeta _{i2}|\varvec{\zeta }_{-i2}) = \sum _{k \sim i} c_{22} \zeta _{k2} + \sum _{k \sim i, ~k <i} c_{21} \left( { \sigma _2 \over \sigma _1} \right) \zeta _{k1} + \sum _{k \sim i, ~k >i} c_{12} \left( { \sigma _1 \over \sigma _2} \right) \zeta _{k1} + \rho ^c_{21} \left( { \sigma _2 \over \sigma _1} \right) \zeta _{i1}.\nonumber \\ \end{aligned}$$

(11)

Notice that both $c_{12}$ and $c_{21}$ are in (10) and (11), respectively. In other words, the spatial parameters $c_{jl}$ and $c_{lj}$ together characterize the conditional cross-spatial dependencies of the jth variable on the lth variable and vice versa. As such, difference between $c_{jl}$ and $c_{lj}$ should be interpreted with caution. For example, evidence of $c_{jl} > c_{lj}$ does not necessarily suggest evidence that the conditional spatial dependency of the jth variable on the lth variable is higher than that of the lth variable on the jth variable, or vice versa. In addition, the asymmetric cross-dependencies are not label-invariant with respect to the labeling of lattice sites (MacNab 2016a, b). In other words, changes to the site-labeling may lead to changes in the estimates of $\{ c_{jl}, j,l = 1,2,\ldots ,p, j \ne l, \}$.

Expressions (10) and (11) also show that the $\beta$-coefficients in MCAR (9) conditionals are functions of the spatial and scale parameters, an “entanglement” issue to be further discussed in Sect. 3.2. In addition, within the Sain et al. framework, multivariate generalization to the LCAR construction is not readily available. The main reason seems to be the “entanglement” issue; recall that the LCAR site-specific conditional variances are functions of both the scale and spatial parameters.

3.2 Separation, separation strategy, and separability

Separation and separation strategy are commonly discussed in the covariance modeling and estimation literature (see Pourahmadi (2011) for a recent review). For example, Barnard et al. (2000) propose a well-known strategy for modeling a covariance matrix in terms of its standard deviations and correlations. The basic idea is to decompose a covariance matrix, say, ${\varSigma }$, into a product of its “variance” component, the standard deviation matrix S, and its “dependence” component, the correlation matrix R: ${\varSigma }= S R S$, the well-known variance-correlation decomposition (Pourahmadi 2011). Barnard et al. call this decomposition a separation strategy because the decomposition is a separation of the standard deviation and correlation matrices. In the context of Bayesian estimation of ${\varSigma }$, for example, such separation enabled Barnard et al. to consider prior specification $p(S, R)=p(S)p(R|S)$, or to assume that S and R are independent and consider their prior specifications separately.

The Sain et al. MCAR construction (9), as well as the CAR and adaptive CAR constructions discussed in Sect. 2, represent a variance-correlation decomposition. This decomposition is also a separation strategy of Barnard et al. (2000). We call (9) a Type I decomposition hereafter. Notice that, in MCAR (9) construction, the ${\varvec{S}}(\varvec{\rho }^c, {\varvec{C}}) = {\varvec{D}}_m \otimes {\varvec{I}}_{p} - {\varvec{I}}_n \otimes \varvec{\rho }^c - ({\varvec{W}}_U \otimes {\varvec{C}}+{\varvec{W}}_U^{\top } \otimes {\varvec{C}}^{\top })$ is expressed by additive sub-components of conditional spatial, cross-spatial, and non-spatial dependencies. As a result, constraints may be considered for ${\varvec{C}}$, independent of the non-spatial parameters ($\varvec{\sigma }$ and those in $\varvec{\rho }^c$), to ensure ${\varvec{S}}(\varvec{\rho }^c, {\varvec{C}})>0$ for valid models (also see Sect. 6). Bayesian estimation of these parameters may be implemented by placing separate priors on the spatial parameters in ${\varvec{C}}$, the partial correlation parameters in $\varvec{\rho }^c$, and the scale parameters $\varvec{\sigma }$.

In the present paper, we also discuss proposals of decomposition and separation (or separation strategy) as a means to manage the “entanglement” and “enforcement” challenges. We show that a separation strategy may aim for a separation of spatial and non-spatial parameters. Consequently, the positivity condition may be more readily enforced by placing constraints on the spatial parameters, independent of the non-spatial ones. Such separation is also discussed to facilitate Bayesian estimation of the spatial and non-spatial parameters.

In multivariate spatial and spatiotemporal statistics, and in multivariate geostatistics in particular, the concept of separability is typically discussed in relation to covariance matrix and cross-covariance functions (Gelfand et al. 2004; Genton 2007; Genton and Kleiber 2015). For modeling multivariate or spatial-temporal processes, for example, a multivariate or spatiotemporal covariance model is said to be separable if the covariance matrix can be written as a tensor product of a spatial covariance matrix and a non-spatial covariance matrix. In other words, a separable model separates spatial dependence from non-spatial dependence. This notion of separability is also part of the matrix-variate distribution theory for covariance models and is often discussed in the context of analysis of matrix-valued or multi-way data (Dawid 1981; Hoff 2011; Martinez-Beneito 2013; Martinez-Beneito et al. 2017).

The Sain et al. (2011) framework does not contain separable models in its family of MGMRFs. However, separable MGMRFs form an important class of multivariate models with many advantages. For example, they are readily available for all univariate GMRFs, readily interpretable, and often easy to implement. A distinct feature of a separable p-variate GMRF is that a precision/covariance matrix of np-dimension is broken down into two precision/covariance matrices of n- and p-dimension. A separable model is often noted for its computational advantages, say, as a more stable and parsimonious alternative to its non-separable counterparts (Hoff 2011). Some advantages of separable models are discussed in Brown et al. (1994), Sun et al. (1998), Banerjee et al. (2014), Conti and O’Hagan (2010) and Pourmohamad and Lee (2016).

Even though separability concerns with separation of the spatial and non-spatial associations that are characterized by the MGMRF precision and covariance matrices, the separation does not disentagle the spatial and non-spatial components nor undo the “entanglement” impact on the MGMRF characterization (see Sects. 3.3 and 3.4). Nevertheless, a separable model is often noted for its computational advantages, say, as a stable and parsimonious alternative to its non-separable counterparts (Hoff 2011).

3.3 Multivariate CARs defined by multivariate conditionals

The Mardia (1988) framework represents a family of MGMRFs $\hbox {vec}(\varvec{\zeta }^{\top }) \sim \hbox {MVN}~({\varvec{0}}, \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })})$ defined by the following full conditionals

$$\begin{aligned} \hbox {E}(\varvec{\zeta }_{i \varvec{.}} | \varvec{\zeta }_{-i \varvec{.}}) = \sum _{k=1}^n w_{ik} \varvec{\beta }_{ik}\varvec{\zeta }_{k \varvec{.}}= \sum _{k: k \sim i} \varvec{\beta }_{ik}\varvec{\zeta }_{k \varvec{.}}, ~ \hbox {Prec}(\varvec{\zeta }_{i \varvec{.}} | \varvec{\zeta }_{-i \varvec{.}}) = \varvec{{\varGamma }}_i, \end{aligned}$$

(12)

provided that $\varvec{{\varGamma }}_i \varvec{\beta }_{ik} = \varvec{\beta }_{ki}^{\top }~ \varvec{{\varGamma }}_k$ (for symmetry) and $\varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })} = {\varvec{T}}({\varvec{I}}_{np} - \varvec{\beta }) > 0$ (for positivity), ${\varvec{T}}= \hbox {Bdiag}(\varvec{{\varGamma }}_1, \ldots , \varvec{{\varGamma }}_n)$ and $\varvec{\beta }=\hbox {Block}(\varvec{\beta }_{ik} )$. The Mardia family of MCARs is denoted $\hbox {MCAR}_{\tiny \hbox {MC}}$ hereafter, where “MC” stands for multivariate conditionals.

The MCAR $\beta$-coefficients in (12) postulate conditional spatial and cross-spatial dependencies; conditional dependencies between variables at co-locations are often postulated by letting $\varvec{{\varGamma }}_i=m_i \varvec{{\varGamma }}$, where $\varvec{{\varGamma }}$ is a p by p precision matrix, $\varvec{{\varGamma }}=\varvec{{\varSigma }}^{-1}$ (MacNab and Gustafson 2007), or $\varvec{{\varGamma }}_i=m_i \varvec{{\varSigma }}^{-1}$ (Gelfand and Vounatsou 2003; Sain and Cressie 2007). Conditional spatial and cross-spatial dependencies are induced and defined by ${\varvec{T}}\varvec{\beta }$.

Within this framework, any univariate CAR has its separable MCAR readily available, say, $\varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}(\varvec{{\varOmega }}_s, \varvec{{\varSigma }}) =\varvec{{\varOmega }}_s \otimes \varvec{{\varGamma }}$, where $\varvec{{\varOmega }}_s$ is a CAR precision matrix. For example, by letting, in (12),

$$\begin{aligned} \varvec{{\varGamma }}_i = m_i \varvec{{\varGamma }}, ~~\varvec{\beta }_{ik} = c m_i^{-1} w_{ik} {\varvec{I}}_p, \end{aligned}$$

one can derive a multivariate generalization of the CAR (3) construction with the following precision and covariance matrices, respectively:

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}(c, \varvec{{\varGamma }}) =({\varvec{D}}_m-c {\varvec{W}}) \otimes \varvec{{\varGamma }},~~\varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}(c, \varvec{{\varSigma }}) =({\varvec{D}}_m-c {\varvec{W}})^{-1} \otimes \varvec{{\varSigma }}.\nonumber \\ \end{aligned}$$

(13)

In addition, letting $c=1$ and $m_i=w_{i+}$ in (13) leads to the well-known improper intrinsic MCAR with a singular precision matrix $\varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}^{\tiny \hbox {MiCAR}}(\varvec{{\varGamma }}) =({\varvec{D}}_w- {\varvec{W}}) \otimes \varvec{{\varGamma }}$; also see MacNab and Gustafson (2007) for a separable multivariate construction of the LCAR.

The Mardia framework contains non-separable models with variable-specific spatial parameters or with the previously mentioned asymmetric matrix ${\varvec{C}}$ of spatial parameters. In Sain and Cressie (2007), for example, a non-separable MGMRF parameterized by ${\varvec{C}}$ and $\varvec{{\varSigma }}$ is formulated through a square root factorization of the covariance matrix $\varvec{{\varSigma }}=\varvec{{\varSigma }}^{1/2}\varvec{{\varSigma }}^{1/2}$, where $\varvec{{\varSigma }}^{1/2}=(\varvec{{\varSigma }}^{1/2})^{\top }$.

Consider a multivariate generalization of the CAR (3) construction. If we were to let, in (12),

$$\begin{aligned} \varvec{{\varGamma }}_i = m_i \varvec{{\varGamma }},~~\varvec{\beta }_{ik} = m_i^{-1} w_{ik} {\varvec{C}}, \end{aligned}$$

the symmetry condition would lead to an inconvenient constraint ${\varvec{C}}\varvec{{\varGamma }}= \varvec{{\varGamma }}{\varvec{C}}^{\top }$. But, if we let ${\varvec{B}}={\varvec{C}}\varvec{{\varGamma }}$ and allow ${\varvec{B}}$ be asymmetric (${\varvec{C}}\varvec{{\varGamma }}\ne \varvec{{\varGamma }}{\varvec{C}}^{\top }$ even when ${\varvec{C}}$ is a diagonal matrix), a non-separable MCAR construction can be built to have the following precision matrix:

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}^{\tiny \hbox {MCAR}_{\hbox {MC}}}({\varvec{C}}, \varvec{{\varGamma }}) = {\varvec{D}}_m \otimes \varvec{{\varGamma }}- ({\varvec{W}}_U \otimes {{\varvec{B}}} + {\varvec{W}}_U^{\top }\otimes {{\varvec{B}}}^{\top }), \end{aligned}$$

(14)

provided that (14) is positive definite. There is more than one way to formulate the p-variate full conditionals that give rise to (14); the Sain and Cressie (2007) proposal is one, which led to (14) with ${\varvec{B}}={\varvec{C}}\varvec{{\varGamma }}$, say, by letting in (12), $\varvec{\beta }_{ik} = m_i^{-1} \varvec{{\varSigma }}^{1/2} \varvec{{\varLambda }}\varvec{{\varGamma }}^{1/2} w_{ik}$, $\varvec{\beta }_{ki} = m_i^{-1} \varvec{{\varSigma }}^{1/2} \varvec{{\varLambda }}^{\top } \varvec{{\varGamma }}^{1/2} w_{ik}$, $i<k$, $\varvec{{\varSigma }}^{1/2}\varvec{{\varSigma }}^{1/2}=\varvec{{\varSigma }}$, $\varvec{{\varGamma }}^{1/2} = \varvec{{\varSigma }}^{-1/2}$, $\varvec{{\varLambda }}= \varvec{{\varGamma }}^{1/2} {\varvec{C}}\varvec{{\varSigma }}^{1/2}$. To enforce the positivity condition on (14) by diagonal dominance restriction, complex constraints on ${\varvec{B}}={\varvec{C}}\varvec{{\varGamma }}$ or $\varvec{{\varLambda }}= \varvec{{\varGamma }}^{1/2} {\varvec{C}}\varvec{{\varSigma }}^{1/2}$ are required. This entanglement of the spatial and non-spatial parameters leads to complex parameter constraints and considerable computational complexity, as noted in Sain and Cressie (2007).

Alternatively, if we let ${\varvec{B}}= \varvec{{\varGamma }}^{1/2}{\varvec{C}}\varvec{{\varGamma }}^{1/2}$ in (14), that is, let in (12) $\varvec{{\varGamma }}_i = m_i \varvec{{\varGamma }}$, and, for $i<k$,

$$\begin{aligned} \varvec{\beta }_{ik} = m_i^{-1}w_{ik} \varvec{{\varSigma }}^{1/2} {\varvec{C}}\varvec{{\varGamma }}^{1/2},~~ \varvec{\beta }_{ki} = m_k^{-1} w_{ik} \varvec{{\varSigma }}^{1/2} {\varvec{C}}^{\top } \varvec{{\varGamma }}^{1/2}, \end{aligned}$$

we can derive a MGMRF construction with precision and covariance matrices:

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}^{\tiny \hbox {MCAR}_{\hbox {MC}}}({\varvec{C}}, \varvec{{\varGamma }}^{1/2})= & {} ({\varvec{I}}_n \otimes \varvec{{\varGamma }}^{1/2}) {\varvec{S}}({\varvec{C}}) ({\varvec{I}}_n \otimes \varvec{{\varGamma }}^{1/2}), ~~ \hbox {and} \nonumber \\ \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}^{\tiny \hbox {MCAR}_{\hbox {MC}}}({\varvec{C}}, \varvec{{\varSigma }}^{1/2})= & {} ({\varvec{I}}_n \otimes \varvec{{\varSigma }}^{1/2}) {\varvec{S}}({\varvec{C}})^{-1} ({\varvec{I}}_n \otimes \varvec{{\varSigma }}^{1/2}), \end{aligned}$$

(15)

provided that ${\varvec{S}}({\varvec{C}}) >0$, where ${\varvec{S}}({\varvec{C}}) = {\varvec{D}}_m \otimes {\varvec{I}}_{p} - ({\varvec{W}}_U \otimes {\varvec{C}}+ {\varvec{W}}_U^{\top } \otimes {\varvec{C}}^{\top })$.

An appeal of the MCAR (15) construction is that the joint precision and covariance matrix is a decomposition that separates spatial parameters in $S( {\varvec{C}})$ from non-spatial parameters in $\varvec{{\varSigma }}^{1/2}$. Consequently, the positivity condition for a valid MGMRF, i.e., ${\varvec{S}}({\varvec{C}}) > 0$, may be enforced by placing constraint on ${\varvec{C}}$, independent of $\varvec{{\varSigma }}^{1/2}$ (see Sect. 6). Expression (15) is named a Type II decomposition hereafter.

Again, due to the entanglement of the spatial and non-spatial components similar non-separable multivariate generalization to the LCAR construction is not readily available.

3.4 A closer look at the Sain et al. and Mardia MGMRF frameworks

A closer look at their respective joint precision and covariance matrices sheds light on their characteristics, and on the differences between the Sain et al. and Mardia frameworks.

Let the Sain et al. MGMRF (9) precision matrix be rewritten as follows:

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}^{\tiny \hbox {MCAR}_{\tiny \hbox {Sain et al. UC}}}({\varvec{C}}, \varvec{\rho }^c, \varvec{\tau }) = {\varvec{D}}_m \otimes \varvec{\tau }- {\varvec{I}}_n \otimes \varvec{\tau }^{1/2} \varvec{\rho }^c \varvec{\tau }^{1/2} - ({\varvec{W}}_U \otimes {\varvec{B}}+ {\varvec{W}}_U^{\top } \otimes {\varvec{B}}^{\top }), \end{aligned}$$

where ${\varvec{B}}= \varvec{\tau }^{1/2} {\varvec{C}}\varvec{\tau }^{1/2}$. It is readily seen that the Sain et al. MGMRF (9) is a non-separable model construction. The Mardia framework contains separable and non-separable models; the framework facilitates formulation of separable models by a separation of its spatial and non-spatial components. Non-separable Mardia MGMRFs can be built by a decomposition of its non-spatial component and by a separation between its spatial component and the decomposition of its non-spatial component.

Recall that ${\varvec{B}}= {\varvec{C}}\varvec{{\varGamma }}$ or ${\varvec{B}}=\varvec{{\varGamma }}^{1/2} {\varvec{C}}\varvec{{\varGamma }}^{1/2}$ in the Mardia MGMRF (14). The elements of ${\varvec{B}}$ characterize conditional spatial and cross-spatial dependencies in the Mardia family of MGMRFs. The associated conditional spatial auto- and cross-correlation functions are

$$\begin{aligned} \hbox {corr}(\zeta _{ij}, \zeta _{kj} |\varvec{\zeta }_{-(ij,kj)})= & {} { B_{jj} \over \tau _j \sqrt{m_i m_k}}, \\ \hbox {corr}(\zeta _{ij}, \zeta _{kl} |\varvec{\zeta }_{-(ij,kl)})= & {} { B_{jl} \over \sqrt{\tau _j \tau _l m_i m_k}}, k > i, \\ \hbox {corr}(\zeta _{ij}, \zeta _{kl} |\varvec{\zeta }_{-(ij,kl)})= & {} { B_{lj} \over \sqrt{\tau _j \tau _l m_i m_k}}, k <i, \end{aligned}$$

where $j, l=1,2,\ldots ,J$, $j \ne l$, $k \sim i$, $\tau _j = \sigma ^{-2}_j$ for the Sain et al. MCARs or $\tau _j = {\varGamma }_{jj}$ for the Mardia MCARs.

The Sain et al. framework was proposed to model multivariate spatial dependencies (Sain et al. 2011). The conditional spatial auto- and cross-correlation functions for the MGMRF (8) are functions of the elements of ${\varvec{C}}$. This MGMRF construction does not imply conditional cross-spatial dependencies if ${\varvec{C}}$ is a diagonal matrix (${\varvec{B}}$ is diagonal when ${\varvec{C}}$ is diagonal). The MGMRF (9) implies positive (or negative) spatial or cross-spatial dependencies when the associated spatial parameters are positive (or negative).

For the Mardia MGMRFs, the matrix ${\varvec{B}}$ serves to interpret the conditional spatial auto- and cross-correlations. A limitation of the these MGMRFs is that the inherent entanglement of the spatial and non-spatial parameters can induce conditional cross-spatial dependencies in undesirable ways. The Mardia MGMRF (14) may entail conditional cross-spatial dependencies even when ${\varvec{C}}$ is a diagonal matrix or ${\varvec{C}}= c {\varvec{I}}_p$ (a separable model), provided that the precision matrix $\varvec{{\varSigma }}$ is not a diagonal matrix. However, the implied conditional cross-spatial dependencies may be unsolicited. This is an issue even for the separable MGMRFs. For example, the separable MGMRF (13) precision matrix has ${\varvec{B}}=c\varvec{{\varGamma }}$, where $\varvec{{\varGamma }}_{jl}$ are often negative, say, when the (partial) correlation between the jth and lth variables at co-locations is positive, which is common in many applications, say, in disease mapping. When c is positive, the separable model implies positive conditional spatial autocorrelations but negative conditional cross-spatial correlations. Similar or more precarious results can arise when ${\varvec{C}}$ is a diagonal or full matrix of spatial parameters.

This limitation of the Mardia family of MGMRFs is not necessarily a problem when a Mardia MCAR model is used for local prediction and spatial smoothing, say, in multivariate disease mapping where only the site-wise local sub-models are deployed. In addition, the entanglement of spatial and non-spatial parameters exerts impact on the MGMRF cross-covariance functions differently. Notice that ${\varSigma }_{jl} >0$ implies positive correlation between the two variables at co-locations. It is readily seen from the separable model covariance matrix (13) that positive correlations between variables at co-locations do not lead to negative cross-covariance functions when $c >0$. Analogous results can be derived for non-separable models with a diagonal matrix of positive spatial parameters.

4 Coregionalization MGMRFs

The basic idea of linear coregionalization for multivariate generalizations of univariate GMRFs is as follows: One can readily produce p-variate coregionalization fields by linear combination of q independent univariate GMRFs, say, each with distinct spatial parameter(s), where $q=p$ or $q \ne p$, in particular $q < p$ for dimension reduction. This is a powerful idea. Any univariate GMRF can have a variety of multivariate generalizations readily derived (MacNab 2016a). In addition, MGMRFs can be produced by linear combination of multivariate latent components (Jin et al. 2007; MacNab 2016a, b).

Specifically, let $\varvec{\eta }=(\eta _{ij})$ denote an n by p matrix of spatially interacting latent variates defined over a finite lattice system and with similar notations $\varvec{\eta }_{\varvec{.}j} \hbox {s}$, $\varvec{\eta }_{i \varvec{.}} \hbox {s}$, $\hbox {vec}(\varvec{\eta })$, and $\hbox {vec}(\varvec{\eta }^{\top })$. A coregionalization MGMRF may be formulated by defining $\varvec{\zeta }$ to be a linear combination of $\varvec{\eta }$ (Jin et al. 2007; MacNab 2016a, b):

$$\begin{aligned} \hbox {vec}(\varvec{\zeta }) =({\varvec{A}}~ \otimes {\varvec{I}}_n) \hbox {vec}(\varvec{\eta }), ~ \hbox { equivalently}, ~ \varvec{\zeta }= \varvec{\eta }{\varvec{A}}^{\top }, \end{aligned}$$

(16)

or

$$\begin{aligned} \hbox {vec}(\varvec{\zeta }^{\top } ) =({\varvec{I}}_n~ \otimes {\varvec{A}}) \hbox {vec}(\varvec{\eta }^{\top }), ~ \hbox {equivalently}, ~ \varvec{\zeta }^{\top } = {\varvec{A}}\varvec{\eta }^{\top }, \end{aligned}$$

(17)

provided that ${\varvec{A}}{\varvec{A}}^{\top } =\varvec{{\varSigma }}> 0$ is a full-rank covariance matrix, where ${\varvec{A}}$ is the p by p linear coregionalization coefficients matrix.

Multivariate coregionalization fields can be built in at least three ways. First, the linear coregionalization framework enables formulations of MGMRFs as linear combinations of independent latent GMRFs via (16); this is discussed further in Sect. 4.1. Multivariate GMRF can be built by linear combination of underlying multivariate latent components via (17); they are presented in Sect. 4.2.

In Sect. 4.3, we present a third way of building MGMRFs via

$$\begin{aligned} \hbox {vec}(\varvec{\zeta }) = {\varvec{H}}\hbox {vec}(\varvec{\eta }), ~\hbox {or,}~ \hbox {vec}(\varvec{\zeta }^{\top } ) = {\varvec{H}}\hbox {vec}(\varvec{\eta }^{\top }), \end{aligned}$$

(18)

provided that ${\varvec{H}}$ is a np by np matrix and $\varvec{{\varSigma }}_{\tiny {\varvec{H}}}= {\varvec{H}}{\varvec{H}}^{\top } >0$ is a full-rank covariance matrix of np-dimension. This is a flexible way to build MGMRFs, say, adding spatially varying coregionalization fields and spatially adaptive coregionalization fields to the collection of coregionalization MGMRFs. Notice that (16) and (17) are special cases of (18).

4.1 The basic coregionalization framework: multivariate coregionalization fields formulated by linear combination of independent latent fields

An important feature of the basic linear coregionalization framework is its flexibility for multivariate generalizations of any univariate GMRF. This can be readily achieved by representing a multivariate coregionalization GMRF via linear combination of independent underlying latent GMRFs with or without unknown scale parameter(s), say, $\varvec{\eta }_{.j} \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varOmega }}_{\varvec{\eta }_{.j}}(c_j))$ without unknown scale parameter(s). The resulting models are denoted $\hbox {cMCAR}_{\tiny \hbox {Ind CARs}}$ hereafter. For example, by letting $\varvec{\eta }_{.j} \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varOmega }}(c_j))$, the resulting coregionalization MCAR construction has the following precision and covariance matrices (MacNab 2016a, b):

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {cMCAR}_{\tiny \hbox {Ind CARs}}}({\varvec{c}}, {\tilde{\varvec{{\varGamma }}}})= & {} ({\tilde{\varvec{{\varGamma }}}} \otimes {\varvec{I}}_n) ~\hbox {Bdiag} (\varvec{{\varOmega }}(c_1), \ldots , \varvec{{\varOmega }}(c_p))~ ({\tilde{\varvec{{\varGamma }}}} \otimes {\varvec{I}}_n)^{\top }, \nonumber \\ \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {cMCAR}_{\tiny \hbox {Ind CARs}}}({\varvec{c}}, {\varvec{A}})= & {} ({\varvec{A}}\otimes {\varvec{I}}_n) ~\hbox {Bdiag} (\varvec{{\varOmega }}(c_1)^{-1}, \ldots , \varvec{{\varOmega }}(c_p)^{-1})~ ({\varvec{A}}\otimes {\varvec{I}}_n)^{\top },\qquad \end{aligned}$$

(19)

where ${\tilde{\varvec{{\varGamma }}}}=({\varvec{A}}^{-1})^{\top }$. Locally adaptive constructions can be readily derived by replacing $c_j$ by ${\varvec{c}}_j=(c_{1j}, c_{2j},\ldots , c_{nj})$ in (19).

It is readily seen from (19) that several cMCAR models may be considered. By defining ${\varvec{A}}$ as the lower triangular Cholesky decomposition or the symmetric square root factorization of $\varvec{{\varSigma }}$, we have order-dependent or order-free cMCAR models (see Jin et al. 2007; MacNab 2016b). In addition, cMCARs can also be defined by letting ${\varvec{A}}$ be an arbitrary p by p matrix, provided that ${\varvec{A}}{\varvec{A}}^{\top }=\varvec{{\varSigma }}>0$ is a covariance matrix (see details in Botella-Rocamora et al. 2015; MacNab 2016b). These cMCAR models are order-free and named M-models hereafter.

Furthermore, some p-variate coregionalization models can be built by (16), in which the coregionalization coefficients matrix ${\varvec{A}}$ is a p by q matrix of specific reparameterization, $q \ne p$, provided that ${\varvec{A}}{\varvec{A}}^{\top }=\varvec{{\varSigma }}>0$ is a covariance matrix. For example, shared component models can be derived for $q > p$ or $q <p$. When p is small, a p-variate shared component model of q components can be formulated with a p by q matrix ${\varvec{A}}$. To give an example, let us consider the following shared component model formulation (Knorr-Held and Best 2001):

$$\begin{aligned} \varvec{\zeta }_{\varvec{.}j} = \varvec{\eta }_{\varvec{.}j} + \delta _j \varvec{\eta }_{\varvec{.}(p+1)} , ~ \varvec{\eta }_{.j} \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varOmega }}(c_j, \sigma _j)), ~ \sum _{j=1}^p \log (\delta _j)= 0. \end{aligned}$$

Its cMCAR model equivalent can be formulated by letting $\hbox {vec}(\varvec{\zeta }) =({\varvec{A}}~ \otimes {\varvec{I}}_n) \hbox {vec}(\varvec{\eta })$, with the following coregionalization coefficients matrix and its associated non-spatial covariance matrix, respectively:

$$\begin{aligned} {\varvec{A}}= & {} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} 1 &{} 0 &{} \cdot \cdot \cdot &{} 0 &{} \delta _1 \\ 0 &{} 1 &{} \cdot \cdot \cdot &{} 0 &{} \delta _2 \\ \cdot &{} \cdot &{} &{} \cdot \\ \cdot &{} \cdot &{} &{} \cdot \\ \cdot &{} \cdot &{} \cdot \cdot \cdot &{} \cdot \\ 0 &{} 0 &{} \cdot \cdot \cdot &{} 1 &{} \delta _p \\ \end{array} \right] _{p \times (p+1)}, \\ \varvec{{\varSigma }}= & {} {\varvec{A}}{\varvec{A}}^{\top } = \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} 1+\delta _1^2 &{} \delta _1 \delta _2 &{} \cdot \cdot \cdot &{} \delta _1 \delta _p\\ \delta _1 \delta _2 &{} 1+\delta _2^2 &{} \cdot \cdot \cdot &{} \delta _2 \delta _p\\ \cdot &{} \cdot &{} &{} \cdot \\ \cdot &{} \cdot &{} &{} \cdot \\ \cdot &{} \cdot &{} \cdot \cdot \cdot &{} \cdot \\ \delta _1 \delta _p &{} \delta _2 \delta _p &{} \cdot \cdot \cdot &{} 1+\delta _p^2 \\ \end{array} \right] _{p \times p}. \end{aligned}$$

Notice that the shared component coregionalization with $q > p$ needs careful specification in order to be identified (MacNab 2010, 2014). A limitation of the $q \ge p$ linear coregionalization is that it is computationally prohibitive when p is large. The number of unknown parameters in the coregionalization coefficients matrix ${\varvec{A}}$ grows quickly, say, by the order of $p(p+1)/2$ when $q=p$.

On the other hand, a major strength of the basic linear coregionalization method is its use in dimension reduction for large p, where the number of latent components $q<p$, or $q \ll p$ (meaning “as few as possible”). For example, the basic linear coregionalization with $q \ll p$ may be a useful tool for multivariate analysis, such as latent variable or component or factor analysis.

4.2 A general linear coregionalization framework: cMCAR models with independent or correlated underlying latent components

The linear coregionalization methods discussed in Jin et al. (2007) and MacNab (2016a, b) are also concerned with formulating cMCARs with underlying (correlated) latent components characterized by the following univariate conditionals:

$$\begin{aligned} \hbox {E}(\eta _{ij}|\varvec{\eta }_{-ij}) = \sum _{k \sim i} \beta _{ijk j} ~\eta _{k j} ~+ \sum _{ l \ne j~k \sim i} \beta _{ijkl}~ \eta _{kl},~ \hbox {Prec}(\eta _{ij}|\varvec{\eta }_{-ij})= m_{i}. \end{aligned}$$

(20)

The main appeal of (20) is that it only implies conditional spatial and cross-spatial dependencies. The non-spatial site-wise dependencies between variables, and the scale parameters, are introduced into the coregionalization field via linear coregionalization (16) or (17) (MacNab 2016a, b). Free of the “entanglement” issue, flexible latent MCAR construction and parameterization may be more readily considered, with intuitively plausible interpretations of correlated latent components in terms of (conditional) spatial dependencies and cross-spatial dependencies.

For example, if we let, in (20), $\beta _{ijkj}=c_{jj} m_i^{-1}w_{ik}$, $\beta _{ijkl}=c_{jl} m_i^{-1}w_{ik}$, $\beta _{kjil}=c_{lj} m_i^{-1}w_{ki}$, $k >i$, where ${\varvec{C}}$ is an asymmetric matrix of spatial parameters, we can derive $\varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\eta }^{\top })}({\varvec{C}}) = {\varvec{S}}({\varvec{C}})$, where ${\varvec{S}}({\varvec{C}}) = {\varvec{D}}_m \otimes {\varvec{I}}_{p} - ( {\varvec{W}}_U \otimes {\varvec{C}}+ {\varvec{W}}_U^{\top } \otimes {\varvec{C}}^{\top })$. The resulting cMCAR construction, denoted $\hbox {cMCAR}_{\tiny \hbox {UC}}$, has the following covariance matrix

$$\begin{aligned} \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}^{\tiny \hbox {cMCAR}_{\tiny \hbox {UC}}}({\varvec{C}}, {\varvec{A}}) = ({\varvec{I}}_n \otimes {\varvec{A}}) {\varvec{S}}({\varvec{C}})^{-1} ({\varvec{I}}_n \otimes {\varvec{A}})^{\top }. \end{aligned}$$

(21)

Expression (21) is a multivariate generalization of the CAR construction (3). When ${\varvec{A}}$ is a full-rank symmetric matrix, the cMCAR (21) and the Mardia MCAR (15) constructions have identical precision and covariance matrices.

In addition, the Mardia (1988) framework (12) offers an option for characterizing underlying latent components:

$$\begin{aligned} \hbox {E}(\varvec{\eta }_{i \varvec{.}} | \varvec{\eta }_{-i \varvec{.}}) = \sum _{k: k \sim i} \varvec{\beta }_{ik}\varvec{\eta }_{k \varvec{.}}, ~ \hbox {Prec}(\varvec{\eta }_{i \varvec{.}} | \varvec{\eta }_{-i \varvec{.}}) = {\varvec{m}}_i {\varvec{I}}_p. \end{aligned}$$

(22)

For example, a coregionalization MCAR equivalent of the Mardia MCAR (15) can be formulated via (22) for the latent MCAR and (17) for the coregionalization MCAR, denoted $\hbox {cMCAR}_{\tiny \hbox {MC}}$ hereafter. The cMCAR models derived via (20) and (22), respectively, have identical precision and covariance matrices. The main differences between the two approaches could be computational. For example, for Bayesian estimation of the cMCARs via MCMC simulations using Gibbs sampling, the latent components modeled by (20) may be simulated through site- and component-wise univariate updates whereas those of MCAR (22) can be sampled by site-wise p-vector updates.

Further, the Sain et al. (2011) (8) framework can be used to formulate MCARs for multivariate latent field $\varvec{\eta }$:

$$\begin{aligned} \hbox {E}(\eta _{ij}|\varvec{\eta }_{-ij}) = \sum _{k \sim i} \beta _{ijkj} \eta _{kj} + \sum _{k \sim i,~ l \ne j} \beta _{ijkl} \eta _{kl} + \sum _{l \ne j} \beta _{ijil} \eta _{il},~~ \hbox {Prec}(\eta _{ij}|\varvec{\eta }_{-ij})= m_{i}.\nonumber \\ \end{aligned}$$

(23)

Any MCAR construction developed within the Sain et al. (2011) framework has a linear coregionalization equivalent, denoted $\hbox {cMCAR}_{\tiny \hbox {Sain et al. UC}}$ hereafter, which can be readily derived via (23) for its latent multivariate field and via (17) for the coregionalization field. One obvious example is to model the latent field by a MCAR with the precision matrix $\varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\eta }^{\top })}({\varvec{C}}, \varvec{\rho }^c) = {\varvec{S}}({\varvec{C}}, \varvec{\rho }^c) ={\varvec{D}}_m \otimes {\varvec{I}}_{p} - {\varvec{I}}_n \otimes \varvec{\rho }^c - ({\varvec{W}}_U \otimes {\varvec{C}}+{\varvec{W}}_U^{\top } \otimes {\varvec{C}}^{\top })$. The linear transformation via $\hbox {vec}(\varvec{\zeta }^{\top } ) =({\varvec{I}}_n \otimes {\varvec{A}}) \hbox {vec}(\varvec{\eta }^{\top })$, with ${\varvec{A}}= \hbox {diag}(\sigma _1, \ldots , \sigma _p)$, leads to a $\hbox {cMCAR}_{\tiny \hbox {Sain et al. UC}}$ construction with the following covariance matrix:

$$\begin{aligned} \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta }^{\top })}^{\tiny \hbox {cMCAR}_{\tiny \hbox {Sain et al. UC}}} ({\varvec{C}}, {\varvec{A}}) = ({\varvec{I}}_n \otimes \varvec{\sigma }) {\varvec{S}}(\varvec{\rho }^c, {\varvec{C}})^{-1} ({\varvec{I}}_n \otimes \varvec{\sigma }), \end{aligned}$$

(24)

which is a linear coregionalization recast of the MCAR (9). By letting $m_i=p w_{i+}+p-1, \forall i$, in (24), we can derive a linear coregionalization recast of the p-fold CAR, denoted $\hbox {cMCAR}_{\tiny p\hbox {-fold}}$ hereafter.

The Sain et al. MCARs and their coregionalization recasts are identical models. A noteworthy difference between the two is computational (discussed in Sect. 7).

4.3 Coregionalization MGMRFs built via $\hbox {vec}(\varvec{\zeta }) = {\varvec{H}}\hbox {vec}(\varvec{\eta })$ or $\hbox {vec}(\varvec{\zeta }^{\top } ) = {\varvec{H}}\hbox {vec}(\varvec{\eta }^{\top })$

The linear coregionalization MCARs of the Type II decomposition are built with a decomposition of the covariance matrix $\varvec{{\varSigma }}$, its non-spatial component. Alternatively, spatially varying coregionalization GMRFs can be built via (18) through a decomposition of its sparse spatial precision matrix or matrices, which may comprise (part of) its spatial components. Here, we illustrate two classes of spatially varying coregionalization models. We also briefly discuss SVC extensions for locally adaptive coregionalization models.

4.3.1 Spatially varying coregionalization constructions: a class of $\hbox {SVC}_{\tiny \hbox {(I)}} \hbox {s}$

A SVC construction, denoted $\hbox {SVC}_{\tiny \hbox {(I)}}$ hereafter, can be formulated by letting $\hbox {vec}(\varvec{\zeta }) = {\varvec{H}}({\varvec{c}}) \hbox {vec}(\varvec{\eta })$, where

• the latent variables $\hbox {vec}(\varvec{\eta })$ are assumed to be correlated only at co-locations, say, they are modeled by a non-spatial multivariate Gaussian field: $\hbox {vec}(\varvec{\eta }) \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varGamma }}\otimes {\varvec{I}}_n)$, $\varvec{{\varGamma }}= \varvec{{\varSigma }}^{-1}$; and

• the spatially varying coregionalization coefficients matrix ${\varvec{H}}({\varvec{c}})$ is defined as ${\varvec{H}}({\varvec{c}})=({\varvec{T}}({\varvec{c}})^{-1})^{\top }$, where ${\varvec{T}}={\varvec{T}}({\varvec{c}})$ is a block diagonal matrix with n by n block matrices ${\varvec{T}}_{jj}(c_j)=\varvec{{\varOmega }}_{\tiny \hbox {CAR}_{j}}(c_j)^{1/2}, \forall j$, and $\varvec{{\varOmega }}_{\tiny \hbox {CAR}_{j}}(c_j)^{1/2}$ is a factorization (say, Cholesky or spectral factorization) of sparse CAR precision matrix $\varvec{{\varOmega }}_{\tiny \hbox {CAR}_j}(c_j)$.

The resulting $\hbox {SVC}_{\tiny \hbox {(I)}}$ construction has the precision and covariance matrices:

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}_{\hbox {(I)}}} = {\varvec{T}}({\varvec{c}}) ~ ( \varvec{{\varGamma }}\otimes {\varvec{I}}_n) ~ {\varvec{T}}({\varvec{c}})^{\top }, ~ \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}_{\hbox {(I)}}} = {\varvec{H}}({\varvec{c}}) ~ ( \varvec{{\varSigma }}\otimes {\varvec{I}}_n) ~ {\varvec{H}}({\varvec{c}})^{\top }, \end{aligned}$$

(25)

where,

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}_{\hbox {(I)}}}= & {} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} {\varGamma }_{11} \varvec{{\varOmega }}_{\tiny \hbox {CAR}_1}(c_1) &{} {\varGamma }_{12} {\tilde{\varvec{{\varOmega }}}}_{12} &{} \cdot \cdot \cdot &{} {\varGamma }_{1p} {\tilde{\varvec{{\varOmega }}}}_{1p}\\ {\varGamma }_{21} {\tilde{\varvec{{\varOmega }}}}_{21}&{} {\varGamma }_{22} \varvec{{\varOmega }}_{\tiny \hbox {CAR}_2}(c_2) &{} \cdot \cdot \cdot &{} {\varGamma }_{2p} {\tilde{\varvec{{\varOmega }}}}_{2p}\\ \cdot &{} \cdot &{} &{} \cdot \\ \cdot &{} \cdot &{} &{} \cdot \\ \cdot &{} \cdot &{} \cdot \cdot \cdot &{} \cdot \\ {\varGamma }_{p1} {\tilde{\varvec{{\varOmega }}}}_{p1} &{} {\varGamma }_{p2}{\tilde{\varvec{{\varOmega }}}}_{p2}&{} \cdot \cdot \cdot &{} {\varGamma }_{pp} \varvec{{\varOmega }}_{\tiny \hbox {CAR}_p}(c_p) \\ \end{array} \right] , \nonumber \\ \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}_{\hbox {(I)}}}= & {} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} {\varSigma }_{11} \varvec{{\varOmega }}_{\tiny \hbox {CAR}_1}(c_1)^{-1} &{} {\varSigma }_{12} {\tilde{\varvec{{\varOmega }}}}_{12} ^{-1} &{} \cdot \cdot \cdot &{} {\varSigma }_{1p} {\tilde{\varvec{{\varOmega }}}}_{1p}^{-1} \\ {\varSigma }_{21} {\tilde{\varvec{{\varOmega }}}}_{21}^{-1} &{} {\varSigma }_{22} \varvec{{\varOmega }}_{\tiny \hbox {CAR}_2}(c_2)^{-1} &{} \cdot \cdot \cdot &{} {\varSigma }_{2p} {\tilde{\varvec{{\varOmega }}}}_{2p}^{-1} \\ \cdot &{} \cdot &{} &{} \cdot \\ \cdot &{} \cdot &{} &{} \cdot \\ \cdot &{} \cdot &{} \cdot \cdot \cdot &{} \cdot \\ {\varSigma }_{p1} {\tilde{\varvec{{\varOmega }}}}_{p1}^{-1} &{} {\varSigma }_{p2}{\tilde{\varvec{{\varOmega }}}}_{p2}^{-1} &{} \cdot \cdot \cdot &{} {\varSigma }_{pp} \varvec{{\varOmega }}_{\tiny \hbox {CAR}_p}(c_p)^{-1} \\ \end{array} \right] ,\qquad \quad \end{aligned}$$

(26)

${\tilde{\varvec{{\varOmega }}}}_{jl} = \varvec{{\varOmega }}_{\tiny \hbox {CAR}_j}(c_j)^{1/2} (\varvec{{\varOmega }}_{\tiny \hbox {CAR}_l}(c_l)^{1/2})^{\top }$, $\varvec{{\varSigma }}=({\varSigma }_{jl})$, and $\varvec{{\varGamma }}=({\varGamma }_{jl})$. It is readily verified that the cross-covariance matrix function $\varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}} [jl]$ in (26) is asymmetric in general, with the exception that ${\tilde{\varvec{{\varOmega }}}}_{jl} = \varvec{{\varOmega }}_{\tiny \hbox {CAR}}(c_j)^{1/2} (\varvec{{\varOmega }}_{\tiny \hbox {CAR}}(c_l)^{1/2})^{\top }$ and the $\varvec{{\varOmega }}_{\tiny \hbox {CAR}}(\varvec{.})^{1/2}$ is the spectral factorization of $\varvec{{\varOmega }}_{\tiny \hbox {CAR}}(\varvec{.})$ or $c_j=c_l$. Expression (25) is a decomposition that separates ${\varvec{T}}({\varvec{c}})$ from $\varvec{{\varGamma }}$ (${\varvec{H}}({\varvec{c}})$ and $\varvec{{\varSigma }}$), which consequently separates ${\varvec{c}}$ (the spatial parameters) from $\varvec{{\varSigma }}$ (the non-spatial parameters). We call (25) a Type III decomposition hereafter.

This SVC formulation is a counterpart of the $\hbox {cMCAR}_{\tiny \hbox {Ind CARs}}$ formulation, the linear coregionalization MCAR formulation represented by the linear combination of independent latent CARs. The two MGMRF constructions produce the same independent GMRFs if and only if $\varvec{{\varSigma }}=\hbox {diag}(\sigma _1^2,\ldots ,\sigma _p^2)$. They also produce the same separable MGMRF when $\varvec{{\varOmega }}_{\tiny \hbox {CAR}_j}(c_j)=\varvec{{\varOmega }}_{\tiny \hbox {CAR}}(c), \forall j$.

However, in general the two approaches produce different multivariate coregionalization constructions. To give an illustrative example, let $\varvec{{\varOmega }}_{\tiny \hbox {CAR}}(c_j) = {\varvec{D}}_m - c_j {\varvec{W}}$ in $\hbox {cMCAR}_{\tiny \hbox {Ind CARs}}$ (19) and in $\hbox {SVC}_{\tiny \hbox {(I)}}$ (26), respectively (i.e., $\varvec{{\varOmega }}_{\tiny \hbox {CAR}}(c_j)$ is the precision matrix of the CAR (3) construction with $\sigma =1$). The resulting $\hbox {SVC}_{\tiny \hbox {(I)}}$ has the following precision and covariance matrices:

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}}= & {} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} {\varGamma }_{11} ({\varvec{D}}_m-c_1{\varvec{W}}) &{} &{} &{} &{} {\varGamma }_{1p} {\tilde{\varvec{{\varOmega }}}}_{1p}\\ &{} \cdot &{} &{} &{} \\ &{} &{} \cdot &{} &{} \\ &{} &{} &{} \cdot &{} \\ {\varGamma }_{p1} {\tilde{\varvec{{\varOmega }}}}_{p1} &{} &{} &{} &{} {\varGamma }_{pp} ({\varvec{D}}_m-c_p {\varvec{W}}) \\ \end{array} \right] , \nonumber \\ \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}}= & {} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} {\varSigma }_{11} ({\varvec{D}}_m-c_1{\varvec{W}}) ^{-1} &{} &{} &{} &{} {\varSigma }_{1p} {\tilde{\varvec{{\varSigma }}}}_{\tiny \hbox {pCAR}}(c_1, c_p) \\ &{} \cdot &{} &{} &{} \\ &{} &{} \cdot &{} &{} \\ &{} &{} &{} \cdot &{} \\ {\varSigma }_{p1}{\tilde{\varvec{{\varSigma }}}}_{\tiny \hbox {pCAR}}(c_p, c_1) &{} &{} &{} &{} {\varSigma }_{pp} ({\varvec{D}}_m-c_p{\varvec{W}})^{-1} \\ \end{array} \right] , \end{aligned}$$

(27)

where ${\tilde{\varvec{{\varOmega }}}}_{\tiny \hbox {pCAR}}(c_j, c_l) = ({\varvec{D}}_m-c_j{\varvec{W}})^{1/2} ( ({\varvec{D}}_m-c_l {\varvec{W}})^{1/2})^{\top }$ and ${\tilde{\varvec{{\varSigma }}}}_{\tiny \hbox {pCAR}}(c_j, c_l)={\tilde{\varvec{{\varOmega }}}}_{\tiny \hbox {pCAR}}(c_j, c_l)^{-1}$. Notice that when $({\varvec{D}}_m-c_j{\varvec{W}})^{1/2}$ is the lower triangular Cholesky square root of ${\varvec{D}}_m-c_j{\varvec{W}}, \forall j$, the cross-covariance matrix functions in (27) are asymmetric functions. However, when $({\varvec{D}}_m-c_j{\varvec{W}})^{1/2}$ is the symmetric square root of ${\varvec{D}}_m-c_j{\varvec{W}}, \forall j$, the cross-covariance functions in (27) are symmetric functions.

The resulting $\hbox {cMCAR}_{\tiny \hbox {Ind CARs}}$, on the other hand, has the following precision and covariance matrices:

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {LMC}}= & {} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} {\sum }_{l=1}^p g_{1l}^2 ({\varvec{D}}_m-c_l {\varvec{W}}) &{} &{} &{} &{} {\sum }_{l=1}^p g_{1l}g_{pl} ({\varvec{D}}_m-c_l {\varvec{W}}) \\ &{} \cdot &{} &{} &{} \\ &{} &{} \cdot &{} &{} \\ &{} &{} &{} \cdot &{} \\ {\sum }_{l=1}^p g_{1l}g_{pl} ({\varvec{D}}_m-c_l {\varvec{W}}) &{} &{} &{} &{} {\sum }_{l=1}^p g_{pl}^2 ({\varvec{D}}_m-c_l {\varvec{W}}) \\ \end{array} \right] ,\nonumber \\ \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {LMC}}= & {} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} {\sum }_{l=1}^p a_{1l}^2 ({\varvec{D}}_m-c_l{\varvec{W}}) ^{-1} &{} &{} &{} &{} {\sum }_{l=1}^p a_{1l}a_{pl} ({\varvec{D}}_m-c_l{\varvec{W}}) ^{-1} \\ &{} \cdot &{} &{} &{} \\ &{} &{} \cdot &{} &{} \\ &{} &{} &{} \cdot &{} \\ {\sum }_{l=1}^p a_{1l}a_{pl} ({\varvec{D}}_m-c_l{\varvec{W}}) ^{-1} &{} &{} &{} &{} {\sum }_{l=1}^p a_{pl}^2 ({\varvec{D}}_m-c_l{\varvec{W}}) ^{-1}\\ \end{array} \right] ,\nonumber \\ \end{aligned}$$

(28)

where ${\tilde{\varvec{{\varGamma }}}}= (g_{jl})$, ${\tilde{\varvec{{\varGamma }}}}=({\varvec{A}}^{-1})^{\top }$, and ${\varvec{A}}=(a_{jl})$. Notice that the cross-covariance matrix functions in (28) are symmetric.

The differences between the two constructions are readily observed. An advantage of the SVC construction over its LMC counterpart is that the SVC models have intuitively more interpretable and, perhaps, more appealing, covariance matrix (27). Specifically, the covariance matrix (27) implies that, if the jth and lth (variable) components of the SVC are independent at co-locations, they are independent at different locations, i.e., ${\varSigma }_{jl}={\varSigma }_{lj}=0$ implies $\varvec{{\varSigma }}_{jl} =\varvec{{\varSigma }}_{lj} =0$. The matrix of cross-covariance functions with respect to the jth and lth variables $\varvec{\zeta }_{\varvec{.}j}$ and $\varvec{\zeta }_{\varvec{.}l}$, the n by n cross-covariance matrix $\varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}} [jl]$, $\forall j \ne l$ in (27), is determined by the co-located covariance ${\varSigma }_{jl}$ between the two variables and the factorizations of the covariance matrices of the associated latent components, the $({\varvec{D}}_m-c_j{\varvec{W}})^{-1/2}$ and $({\varvec{D}}_m-c_l{\varvec{W}})^{-1/2}$. The matrix of variable-specific covariance functions, the n by n covariance matrix $\varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}} [jj]$, $\forall j$, is the product of the variance ${\varSigma }_{jj}$ of the jth SVC variable $\varvec{\zeta }_{\varvec{.}j}$ and the covariance matrix $\varvec{{\varSigma }}_{\tiny \hbox {GMRF}}(c_j)$ of the j latent variable $\varvec{\eta }_{\varvec{.}j}$. In other words, when $c_j > 0$, $\forall j$, the smoothness of the jth component of the SVC (27) is characterized by the smoothness of the jth latent component GMRF$(c_j)$.

On the other hand, when $c_j > 0$, $\forall j$, and ${\varvec{A}}$ is a full matrix in (28), the n by n covariance matrix $\varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {LMC}}[jj]= \sum _{l=1}^p a_{jl}^2 ({\varvec{D}}_m-c_l {\varvec{W}})^{-1}, \forall j,$ implies that the smoothness of any (variable) component in the LMC (28), and in linear coregionalization model in general, say, a LMC with a full matrix ${\varvec{C}}$ of spatial parameters, is restricted to that of the roughest underlying univariate latent component. In multivariate geostatistics literature, this is considered a limitation of the linear coregionalization method (Genton and Kleiber 2015).

In addition, if the ${\varvec{A}}$ in (28) is a lower triangular matrix, for example, the Cholesky factorization of $\varvec{{\varSigma }}$, the diagonal covariance matrices in (28) are

$$\begin{aligned} \varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {LMC}}[jj]= \sum _{l=1}^j a_{jl}^2 ({\varvec{D}}_m-c_l{\varvec{W}}) ^{-1}, ~j=1,2,\ldots ,p. \end{aligned}$$

(29)

From (29) follows that the smoothness of the jth component of the linear coregionalization model is restricted to that of the roughest underlying latent component among GMRF($c_1$), GMRF($c_2$),.., GMRF($c_j$). As a result, the (estimated) LMC has ordered spatial smoothing parameters $c_{1}> c_{2} \cdots > c_{p}$. Analogously we can further ascertain that the (estimated) LMC with a full matrix ${\varvec{C}}$ and a lower triangular matrix ${\varvec{A}}$ should have ordered spatial smoothing parameters $c_{11}> c_{22} \cdots > c_{pp}$.

It is readily seen, from (27) and (28), that entanglement of the spatial and non-spatial parameters is still present but exerts different influence on the SVC (conditional) spatial and cross-spatial correlation functions, compared to the entanglement impact on those of the LMC. Nevertheless, the precision and covariance matrices for the two constructions shed light on the potential utility of the linear and spatially varying coregionalization models: They could be used to model spatially varying covariance and cross-covariance functions and enable spatial smoothing of covariance and cross-covariance functions.

When $m_i=w_{i+}$, $\forall i$, and $\varvec{{\varOmega }}_{\tiny \hbox {CAR}_j}(c_j)^{1/2}$ represents the Cholesky or spectral factorization, $\forall j$, SVC (27) produces a coregionalization recast of the previously mentioned Carlin and Banerjee (2003) MCAR or Gelfand and Vounatsou (2003) MCAR.

Consider the following spectral factorization of the CAR (3) (excluding iCAR ) precision matrix without the scale parameter:

$$\begin{aligned} \varvec{{\varOmega }}_{\varvec{\zeta }}(c) = {\varvec{Q}}\hbox {diag}(\varvec{\lambda }(c, {\varvec{e}}^{*})) {\varvec{Q}}^{\top }, \end{aligned}$$

(30)

provided that $c \ne 1$ and $c \in ({1 / e^{*}_1}, 1 / e^{*}_n)$, where $e^*_1> e^*_2>,\ldots ,>e_n^*$ are the ordered eigenvalues of ${\tilde{{\varvec{W}}}}$, ${\varvec{H}}_{{\tilde{w}}}$ is the n by n matrix of associated eigenvectors, ${\varvec{Q}}={\varvec{D}}_m^{1/2}{\varvec{H}}_{{\tilde{w}}}$, and $\varvec{\lambda }({\varvec{c}}, {\varvec{e}}^{*})= (1-c~ e_1^{*}, \ldots , 1-c~ e_n^{*})$, ${\varvec{e}}^{*} =(e^*_1,\ldots ,e_n^*)$. The decomposition required for ${\varvec{H}}({\varvec{c}})$ in SVC (25) is simplified by letting ${\varvec{H}}_{jj} ={\tilde{{\varvec{Q}}}} \hbox {diag} \varvec{\lambda }(c_j, {\varvec{e}}^{*})^{-1/2}$, where ${\tilde{{\varvec{Q}}}}={\varvec{D}}_m^{-1/2}{\varvec{H}}_{{\tilde{w}}}$, ${\varvec{H}}_{{\tilde{w}}}$ (or ${\tilde{{\varvec{Q}}}}={\varvec{H}}_{{\tilde{w}}}$). The resulting covariance matrix (27) can be written as $\varvec{{\varSigma }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}_{\hbox {(I)}}} = {\varvec{H}}({\varvec{c}}) ~ ( \varvec{{\varSigma }}\otimes {\varvec{I}}_n) ~ {\varvec{H}}({\varvec{c}})^{\top }$:

$$\begin{aligned} {\check{{\varvec{Q}}}} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} {\varSigma }_{11} \hbox {diag}(\varvec{\lambda }(c_1, {\varvec{e}}^{*}))^{-1} &{} &{} &{} &{} {\varSigma }_{1p} \hbox {diag}(\varvec{\lambda }(c_1, {\varvec{e}}^{*}) \varvec{\lambda }(c_p, {\varvec{e}}^{*}))^{-1/2}\\ &{} \cdot &{} &{} &{} \\ &{} &{} \cdot &{} &{} \\ &{} &{} &{} \cdot &{} \\ {\varSigma }_{p1} \hbox {diag}(\varvec{\lambda }(c_1, {\varvec{e}}^{*}) \varvec{\lambda }(c_p, {\varvec{e}}^{*}))^{-1/2} &{} &{} &{} &{} {\varSigma }_{pp} \hbox {diag}(\varvec{\lambda }(c_1, {\varvec{e}}^{*}))^{-1}\\ \end{array} \right] {\check{{\varvec{Q}}}} ^{\top },\nonumber \\ \end{aligned}$$

(31)

where ${\check{{\varvec{Q}}}} = {\varvec{I}}_p \otimes {\tilde{{\varvec{Q}}}}$. The abovementioned spectral decomposition offers a computational option for estimation of the SVC (see Sect. 6). Similar reconstruction and reparameterization may also be considered for the LMC models. Further, for both the linear and spatially varying coregionalization MGMRF constructions, dimension reduction may be considered with respect to both n and p. For example, in some applications (say, disease mapping) where the spatial parameters are typically assumed to be positive for positive spatial autocorrelations (say, for spatial smoothing), one may consider reducing the site-dimension from n to $n_r$, where $n_r$ is the total number of the positive eigenvalues $e^*_1, e^*_2,\ldots ,e_{n_r}^*$.

Extensions to $\hbox {SVC}_{\tiny \hbox {(I)}}$ (25) may be considered for $\hbox {vec}(\varvec{\zeta }^{\top }) = {\varvec{H}}({\varvec{C}}) \hbox {vec}(\varvec{\eta }^{\top })$, where ${\varvec{H}}({\varvec{C}})=({\varvec{T}}({\varvec{C}})^{-1})^{\top }$ and ${\varvec{T}}({\varvec{C}})$ is a decomposition of a MCAR(${\varvec{C}}$) precision matrix, for example, ${\varvec{T}}({\varvec{C}}) {\varvec{T}}({\varvec{C}})^{\top } = {\varvec{S}}({\varvec{C}}) = {\varvec{D}}_m \otimes {\varvec{I}}_{p} - ({\varvec{W}}_U \otimes {\varvec{C}}+{\varvec{W}}_U^{\top } \otimes {\varvec{C}}^{\top })$ and ${\varvec{C}}$ is the previously defined p by p matrix of spatial parameters.

4.3.2 Spatially varying coregionalization constructions: a class of $\hbox {SVC}_{\tiny \hbox {(II)}} \hbox {s}$

The $\hbox {SVC}_{\tiny \hbox {(I)}}$ constructions may be further generalized to formulating SVCs with a latent MGMRF. For example, the latent multivariate Gaussian field in $\hbox {SVC}_{\tiny \hbox {(I)}}$ (25) may be replaced by a separable MGMRF, say, by letting $\varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\eta })}(c_{p+1}, \varvec{{\varGamma }}) = \varvec{{\varGamma }}\otimes \varvec{{\varOmega }}_{\tiny \hbox {CAR}_{p+1}}(c_{p+1})$. The precision matrix of the resulting SVC formulation, denoted $\hbox {SVC}_{\tiny \hbox {(II)}}$, is

$$\begin{aligned} \varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta })}^{\tiny \hbox {SVC}_{\hbox {(II)}}} = {\varvec{T}}({{\varvec{c}}}) ~ ( \varvec{{\varGamma }}\otimes \varvec{{\varOmega }}_{\tiny \hbox {CAR}_{p+1}}(c_{p+1}) ) ~ {\varvec{T}}({{\varvec{c}}})^{\top }. \end{aligned}$$

(32)

This construction represents a separation strategy for separating $~{\varvec{T}}({{\varvec{c}}})$, $\varvec{{\varGamma }}$, and $\varvec{{\varOmega }}_{\tiny \hbox {CAR}_{p+1}}(c_{p+1})$. We call (32) a Type IV decomposition and the resulting construction a $\hbox {SVC}_{\tiny \hbox {(II)}}$ construction.

4.3.3 Locally adaptive coregionalization models

The SVC constructions presented in Sects. 4.3.1 and 4.3.2 can be readily generalized to allow for site-specific spatial parameters in their coregionalization models. For example, spatially varying coregionalization models defined by (25) or (32) can have their locally adaptive SVC extensions readily derived by replacing the ${\varvec{c}}$ with ${\tilde{{\varvec{c}}}}=(c_{11}, c_{21},\ldots ,c_{n1},\ldots , c_{1p}, c_{2p},\ldots ,c_{np})$.

Locally adaptive $\hbox {SVC}_{\tiny \hbox {(I)}}$ or $\hbox {SVC}_{\tiny \hbox {(II)}}$ constructions may also be formulated by letting ${\varvec{T}}= {\varvec{H}}^{-1}$ and ${\varvec{H}}= \hbox {diag}(\sigma _{11}, \ldots , \sigma _{n1},\ldots , \sigma _{1p},\ldots , \sigma _{np})$ in (25) or (32) and by considering CAR or MCAR priors for the log$(\sigma _{ij}$)s.

In addition, adaptive coregionalization MGMRFs can be readily formulated via (18) with ${\varvec{H}}= \hbox {diag}(\sigma _{11}, \ldots , \sigma _{1p},\ldots , \sigma _{n1},\ldots , \sigma _{np})$ and with the underlying latent MGMRFs, represented by $\hbox {vec}(\varvec{\eta }^{\top }) \sim \hbox {MCAR}_{\tiny \hbox {Sain et al. UC}} ({\varvec{C}}, \varvec{\rho }^c)$.

4.4 Linear coregionalization recasts and reconstructions of CARs and priors for orthogonal random effects

By viewing a CAR or adaptive CAR as the simplest case of its multivariate counterpart, one can recast it via coregionalization reconstruction. For example, a coregionalization recast of an adaptive CAR could be derived by letting $\varvec{\zeta }= {\varvec{H}}\varvec{\eta }$, $\varvec{\eta }\sim \hbox {CAR}(c)$ or $\varvec{\eta }\sim \hbox {CAR}({\varvec{c}})$, ${\varvec{H}}= \hbox {diag}(\sigma _{1}, \ldots , \sigma _{n})$, $\varvec{\eta }=(\eta _1, \ldots ,\eta _n)^{\top }$, and $\varvec{\zeta }=(\zeta _1, \ldots ,\zeta _n)^{\top }$. In addition, a coregionalization recast of the CAR (3) (excluding the iCAR) construction can be formulated via the spectral decomposition (30) discussed in Sect. 4.3.1:

$$\begin{aligned} \varvec{\zeta }= {\varvec{H}}\varvec{\eta }, ~~ \varvec{\eta }\sim \hbox {MVN}({\varvec{0}}, \sigma ^{-2} \varvec{\lambda }(c, {\varvec{e}}^*)^{-1}),~~ {\varvec{H}}= {\check{{\varvec{Q}}}}, \end{aligned}$$

(33)

where ${\check{{\varvec{Q}}}}$ and $\varvec{\lambda }(c, {\varvec{e}}^*)$ are defined in Sect. 4.3.1 for expression (31). By letting $\varvec{\eta }\sim \hbox {MVN}({\varvec{0}}, \varvec{\sigma }^{-2} \varvec{\lambda }(c, {\varvec{e}}^*)^{-1})$ or $\varvec{\eta }\sim \hbox {MVN}({\varvec{0}}, \varvec{\sigma }^{-2} \varvec{\lambda }({\varvec{c}}, {\varvec{e}}^*)^{-1})$ in (33), we can derive a coregionalization reconstruction of the adaptive CAR (6). One application of these coregionalization recasts and reconstructions would be to enable exploration of different computational approaches to the implementation of these models (discussed in Sect. 7).

Recall that the iCAR has a precision matrix $\varvec{{\varOmega }}_{\varvec{\zeta }} (\sigma )= \sigma ^{-2} (D_w-{\varvec{W}})$ of rank $n-1$. The iCAR reparameterization of Reich et al. (2006) and Goicoa et al. (2018) for orthogonal random effects can be derived by a coregionalization reconstruction of iCAR via $\varvec{\zeta }= {\tilde{{\varvec{H}}}} \varvec{\eta }$, where ${\tilde{{\varvec{H}}}}$ is a n by $n-1$ matrix of eigenvectors with respect to the ordered nonzero eigenvalues $\varsigma _1> \varsigma _2>, \ldots ,> \varsigma _{n-1}>0$ of ${\varvec{D}}_w-W$, $\varvec{\eta }=(\eta _1, \ldots , \eta _{n-1})^{\top }$, $\eta _i \sim \hbox {N}(0, \sigma _i^{-2})$, $\sigma _i^2 = \sigma ^2 \varsigma _i^{-1}$. The resulting model, denoted $\hbox {iCAR}_{\tiny \hbox {SVC}}$ hereafter, has the following covariance matrix:

$$\begin{aligned} \varvec{{\varSigma }}_{\varvec{\zeta }}^{\tiny \hbox {iCAR}_{\tiny \hbox {SVC}}} (\sigma )= \sigma ^2 {\tilde{{\varvec{H}}}} ~ (\hbox {diag}(\varvec{\varsigma }))^{-1} {\tilde{{\varvec{H}}}}^{\top }, \end{aligned}$$

(34)

where $\varvec{\varsigma }=(\varsigma _1, \ldots ,\varsigma _{n-1})$.

Likewise, the Goicoa et al. (2018) reparameterization of the LCAR can be derived by a similar coregionalization reconstruction. Specifically, because the iCAR and LCAR precision matrices share a common matrix of eigenvectors but have different eigenvalues (Goicoa et al. 2018), the LCAR reconstruction, denoted $\hbox {LCAR}_{\tiny \hbox {SVC}}$ hereafter, has the following covariance matrix:

$$\begin{aligned} \varvec{{\varSigma }}_{\varvec{\zeta }}^{\tiny \hbox {LCAR}_{\tiny \hbox {SVC}}} (\sigma , c)= \sigma ^2 {\tilde{{\varvec{H}}}} ~ (\hbox {diag}({\varvec{c}}(c, \varvec{\varsigma })))^{-1} {\tilde{{\varvec{H}}}}^{\top }, \end{aligned}$$

where ${\varvec{c}}(c, \varvec{\varsigma })=(c \varsigma _1+(1-c), \ldots ,c \varsigma _{n-1}+(1-c))$ and ${\tilde{{\varvec{H}}}}$ is identical to that in (34).

Analogous coregionalization reconstructions for orthogonal random effects priors can be built for both multivariate and modified multivariate BYMs (MacNab 2011) and for multivariate generalizations of LCARs via linear or spatially varying coregionalization.

4.5 MGMRFs for variable-specific lattice systems

A MGMRF defined by univariate conditionals may be formulated for variable-specific lattice-neighbor schemes discussed in Besag (1974). In practice, however, enforcing the positivity condition can be a considerable challenge for MGMRFs with a matrix ${\varvec{C}}$ of spatial parameters. Here, we discuss MGMRFs with a diagonal matrix ${\varvec{C}}_{\tiny \hbox {diag}}$ of spatial parameters. Consider a coregionalization MGMRF built by linear combination of independent GMRFs, or its SVC counterpart, where constraints on variable-specific spatial parameters may be derived with respect to variable-specific lattice-neighbor schemes. For example, the latent GMRFs may be built for different, but similar, lattice systems, say, lattices with the associated neighborhood defined by area adjacency, neighbors defined by first- and second-order connectivity, and neighbors defined by a distance criterion for proximity, etc. This would also widen the scope for MGMRF application, say, in image processing and analysis where competing models of different neighbor schemes and neighbor sets may be considered and model selection may be made (Kashyap and Chellappa 1983).

The abovementioned variable-wise lattice-neighbor schemes may also be considered for a Sain et al. MCAR with a diagonal matrix of spatial parameters. While a Sain et al. MCAR with a diagonal matrix ${\varvec{C}}$ of spatial parameters does not imply conditional cross-spatial dependencies, it does induce covariance models with cross-covariance functions, provided that the non-spatial partial correlation matrix $\varvec{\rho }^c$ implies associations between variables at co-locations.

In addition, as mentioned in Sain et al. (2011), we could also consider joint models of multiple variables defined over different lattices. In other words, a more generalized conceptualization of multivariate lattice data may be explored. Multivariate lattice data do not have to be multivariate observations taken over the same finite lattice representation. Instead, relevant data collected under similar but different spatial settings may be modeled jointly, as in the case of aggregates (such as disease incidence and health outcomes) over spatial grids or geographical regions for some variables and observations attributable to a fix location in each of the grid boxes or geographical regions (say, multivariate measurements of pollutants and air qualities collected over monitoring sites) for others. This broader conceptualization also widens the scope for MGMRF formulation and application over the previously mentioned component-wise lattice-neighbor schemes.

5 The Martinez-Beneito framework and its relation to coregionalization

In the context of multivariate disease mapping, Martinez-Beneito (2013) proposes formulation of MCARs via

$$\begin{aligned} \varvec{\zeta }~ = {\tilde{\varvec{{\varSigma }}}}_w ~\varvec{\epsilon }~{\tilde{\varvec{{\varSigma }}}}_b^{\top }. \end{aligned}$$

(35)

Here, $\varvec{\epsilon }= (\epsilon _{ij})$, $\epsilon _{ij} \sim \hbox {N}(0,1)$ and ${\tilde{\varvec{{\varSigma }}}}_w$ and ${\tilde{\varvec{{\varSigma }}}}_b$ are lower triangular matrices of the Cholesky decompositions of within- and between-variable covariance matrices ${\varvec{{\varSigma }}}_w = {\tilde{\varvec{{\varSigma }}}}_w {\tilde{\varvec{{\varSigma }}}}_w ^{\top }$ and ${\varvec{{\varSigma }}}_b = {\tilde{\varvec{{\varSigma }}}}_b {\tilde{\varvec{{\varSigma }}}}_b ^{\top }$, respectively. The Martinez-Beneito framework is motivated by the idea that zero-mean MCAR models with separable cross-covariance functions can be formulated by performing two transformations to $\varvec{\epsilon }$: a pre-multiplication ${\tilde{\varvec{{\varSigma }}}}_w ~\varvec{\epsilon }$ to induce dependencies among geographic units, and a post-multiplication $({\tilde{\varvec{{\varSigma }}}}_w ~\varvec{\epsilon }) {{\tilde{\varvec{{\varSigma }}}}_b}^{\top }$ to induce dependencies between diseases at co-locations, denoted $\hbox {vec}(\varvec{\zeta }) \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varSigma }}_{\varvec{\zeta }}^{-1})$, and $\varvec{{\varSigma }}_{\varvec{\zeta }} =\varvec{{\varSigma }}_b \otimes \varvec{{\varSigma }}_w$.

Under the Martinez-Beneito (2013) framework, non-separable MCAR models can be formulated via

$$\begin{aligned} \varvec{\zeta }~ = ({\tilde{\varvec{{\varSigma }}}}_w ~\varvec{\epsilon }) ~~{\tilde{\varvec{{\varSigma }}}}_b^{\top } =\varvec{\phi }~ {{\tilde{\varvec{{\varSigma }}}}_b}^{\top }, \end{aligned}$$

(36)

which, when $\varvec{\phi }=\varvec{\eta }~$ and ${\tilde{\varvec{{\varSigma }}}}_b={\varvec{A}}$, has a LMC equivalent vec($\varvec{\zeta }) = ({\varvec{A}}\otimes {\varvec{I}}_n) \hbox {vec}(\varvec{\eta })$, equivalently expressed as $\varvec{\zeta }= \varvec{\eta }{\varvec{A}}^{\top }$. In Martinez-Beneito (2013) and Botella-Rocamora et al. (2015), the column vectors $\varvec{\phi }_{\varvec{.}1}$, $\varvec{\phi }_{\varvec{.}2}$ ,..., $\varvec{\phi }_{\varvec{.}p}$ in (36) are assumed to represent independent CAR components.

In addition, consider post-multiplication first in (35), i.e., let

$$\begin{aligned} \varvec{\zeta }~ = {\tilde{\varvec{{\varSigma }}}}_w ~(\varvec{\epsilon }~~{\tilde{\varvec{{\varSigma }}}}_b^{\top }) ={\tilde{\varvec{{\varSigma }}}}_w~\varvec{\varphi }, \end{aligned}$$

(37)

and $\hbox {vec}(\varvec{\varphi }) \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varSigma }}_b^{-1} \otimes {\varvec{I}}_n)$. Expression (37) is equivalent to the spatially varying coregionalization formulation of vec$(\varvec{\zeta }) = ({\varvec{I}}_p \otimes {\tilde{\varvec{{\varSigma }}}}_w) \hbox {vec}(\varvec{\varphi })$.

6 Estimation of constrained spatial and non-spatial parameters

For the MCARs and coregionalization MCARs discussed herein [with the exception of the $\hbox {MCAR}_{\tiny \hbox {Mardia MC}}$ (14)], when ${\varvec{C}}=\hbox {diag}({\varvec{c}})$ is a diagonal matrix or ${\varvec{C}}=c {\varvec{I}}_p$, the constraint on the elements of ${\varvec{C}}$ for a valid MCAR is the same as that for its associated univariate CAR (Jin et al. 2007; MacNab 2016a).

When ${\varvec{C}}$ is a symmetric matrix, a reparameterization of ${\varvec{C}}$ via its spectral decomposition can be considered (Jin et al. 2007). Let ${\varvec{C}}={\varvec{C}}({\varvec{e}}, \varvec{\theta }) = P(\varvec{\theta }) \hbox {diag}({\varvec{e}})P(\varvec{\theta }) ^{\top }$, where $P(\varvec{\theta })$ is an orthogonal matrix parameterized by $p (p-1)/2$ Givens angles, $\varvec{\theta }=(\theta _1,\ldots ,\theta _p)$, and ${\varvec{e}}=(e_1,e_2,\ldots ,e_p)$, and $e_1> e_2 >, \ldots , e_p$ are the ordered eigenvalues of ${\varvec{C}}$. The orthogonal matrix of eigenvectors in the spectral decomposition is reparameterized by the Givens angles to enforce shrinkage of ${\varvec{C}}$ toward diagonality (Daniels and Kass 1999; MacNab 2016a, b). With this reparameterization, the constraint on ${\varvec{C}}$ for a valid model is simplified to a constraint on its eigenvalues, which is the same sufficient and necessary constraint for its univariate CAR counterpart, i.e., $e_j \in (c_{\tiny \hbox {mim}}, c_{\tiny \hbox {max}})$. Posterior estimation and inference for the elements of ${\varvec{C}}$ may be carried out by placing uniform priors $e_j \sim \hbox {Unif}(c_{\tiny \hbox {mim}}, c_{\tiny \hbox {max}})$ on the unknown eigenvalues and $\theta _j \sim \hbox {Unif}(-\pi /2, \pi /2)$ on the Givens angles (see Daniels and Kass 1999; Jin et al. 2007; MacNab 2016a, b for technical details). It is worth mentioning that $\hbox {cMCAR}_{\tiny \hbox {UC}}({\varvec{C}}({\varvec{e}}, \varvec{\theta }), {\varvec{A}})$ is equivalent to $\hbox {cMCAR}_{\tiny \hbox {UC}}({\varvec{e}}, P(\varvec{\theta }) {\varvec{A}}P(\varvec{\theta })^{\top })$ (Martinez-Beneito 2013). The latter can be derived by the linear combination of independent but ordered latent CAR($e_i$)s, with the linear coregionalization coefficient matrix $P(\varvec{\theta }) {\varvec{A}}P(\varvec{\theta })^{\top }$ (see Martinez-Beneito (2013) and MacNab (2016b) for a more detailed discussion).

When ${\varvec{C}}$ is asymmetric, an analogous approach is the singular value decomposition (Greco and Trivisano 2009; MacNab 2016a): Let ${\varvec{C}}= {\varvec{C}}({\varvec{s}}, \varvec{\theta }_L, \varvec{\theta }_R) = P_L(\varvec{\theta }_L) \hbox {diag}({\varvec{s}})P_R(\varvec{\theta }_R) ^{\top }$, where ${\varvec{s}}=(s_1,s_2,\ldots ,s_p)$ comprises the ordered singular values, $P_L(\varvec{\theta }_L)$ and $P_R(\varvec{\theta }_R)$ represent the corresponding orthogonal matrices, each parameterized by a set of $p(p-1)/2$ Givens angles, denoted $\varvec{\theta }_L$ and $\varvec{\theta }_R$ respectively. Analogous to the spectral decomposition of a symmetric ${\varvec{C}}$, placing uniform priors on the admissible (positive) singular values, say, $s_j \sim \hbox {Unif}(0, c_{\tiny \hbox {max}}), \forall j$, and on the Givens angles leads to the posterior shrinkage estimation of ${\varvec{C}}$ (MacNab 2016a, b).

As an alternative, and also within the context of Bayesian multivariate disease mapping, MacNab (2016b) explored the option of placing hierarchical priors (HPs) directly onto the spatial parameters in ${\varvec{C}}$ for linear coregionalization MCARs of the Type II decomposition. The basic idea of this approach is to use priors as a means of imposing constraints on the parameters in ${\varvec{C}}$. This is done by placing a weakly informative HP on the diagonal elements $c_{jj} \hbox {s}$ of ${\varvec{C}}$, with a HP (say, zero-mean Gaussian HP) on its off-diagonal elements to encourage shrinkage toward a diagonal matrix. Compared to the previously mentioned methods via decomposition and reparameterization of ${\varvec{C}}$, this approach was shown to perform comparably well, resulting in similar posterior prediction and inference for disease risks and showing modestly less posterior shrinkage to ${\varvec{C}}$; see MacNab (2016a, b) for technical details and illustrative examples.

The methods for Bayesian estimation of ${\varvec{C}}$ mentioned above apply to Mardia (15) and Sain et al. (9) MCARs and their respective coregionalization equivalents; see Sect. 8 for illustrative examples and the Supplementary Material for this paper for illustrative WinBUGS examples.

For separable MCARs, a common approach to the Bayesian estimation of $\varvec{{\varSigma }}$ is using the inverse Wishart prior for $\varvec{{\varSigma }}$ (Gelfand and Vounatsou 2003) or the Wishart prior on $\varvec{{\varGamma }}$ (MacNab and Gustafson 2007; MacNab 2009). Alternatively, the estimation of $\varvec{{\varSigma }}$ can be carried out by using the Barnard et al. (2000) variance-correlation decomposition strategy and by placing priors on the standard deviation and correlation parameters. For non-separable Mardia MCARs, a symmetric decomposition of $\varvec{{\varSigma }}=\varvec{{\varSigma }}^{1/2}\varvec{{\varSigma }}^{1/2}$ requires $\varvec{{\varSigma }}^{1/2}$ to be the symmetric square root of $\varvec{{\varSigma }}$ (Harville 2007; MacNab 2016b). For example, in MacNab (2016a, b) and the present paper, we let $\varvec{{\varSigma }}^{1/2}=P(\varvec{\theta }^s) {\varvec{e}}^s P(\varvec{\theta }^s)^{\top }$, where ${\varvec{e}}^s=(e_1^{1/2}, \ldots , e_p^{1/2})$, $e_1>\cdots >e_p$ are the ordered eigenvalues of $\varvec{{\varSigma }}$ and $\varvec{\theta }^s$ is the p-vector of Givens angles.

For non-separable linear coregionalization MCARs, there are options for estimating $\varvec{{\varSigma }}$ or working with the coregionalization coefficient matrix ${\varvec{A}}$, provided that ${\varvec{A}}{\varvec{A}}^{\top }=\varvec{{\varSigma }}$. One option is to let ${\varvec{A}}$ be a lower triangular matrix representing the unique Cholesky decomposition of $\varvec{{\varSigma }}$ (Jin et al. 2007; MacNab 2016a, b). The resulting cMCARs are fully identified but the triangular matrix ${\varvec{A}}$ imposes an order among the variables, leading to order-dependent cMCARs; see MacNab (2016b) for detailed discussions and illustrative examples.

Order-dependent models can have advantages when the variables under consideration are causally related or have a natural ordering (say, in spatiotemporal models). However, minor-to-modest-order sensitivities have been observed in multivariate disease mapping where the ordering of the disease-specific variables in the model might be irrelevant to the predictions of the disease-specific risks (Martinez-Beneito 2013; MacNab 2016b). One option for building order-free cMCARs is to let ${\varvec{A}}$ be the previously defined symmetric square root factorization of $\varvec{{\varSigma }}$ (MacNab 2016b).

Further, the M-based models proposed in Botella-Rocamora et al. (2015), named M-models in MacNab (2016b) and hereafter, are also order-free cMCARs defined by the linear combination of independent CARs, where ${\varvec{A}}$ is a p by p arbitrary matrix, provided that ${\varvec{A}}{\varvec{A}}^{\top } = \varvec{{\varSigma }}$ (Botella-Rocamora et al. 2015). The resulting cMCARs lose identification to ${\varvec{A}}$ and the spatial parameters in the latent CARs; however, they may retain identification to $\varvec{{\varSigma }}={\varvec{A}}{\varvec{A}}^{\top }$ and ${\varvec{B}}^{-1}= {\varvec{A}}{\varvec{C}}_{\tiny \hbox {diag}}^{-1} {\varvec{A}}^{\top }$. The M-model is noted for its computational advantages in the context of Bayesian multivariate disease mapping; see Botella-Rocamora et al. (2015) for technical details and illustrative examples.

7 Multivariate GMRFs as prior models and Bayesian computation

In recent literature, MGMRFs are typically used as prior models in Bayesian hierarchical inferential frameworks with, say, a data model $\left[ {\varvec{Y}}|\varvec{\zeta }, \varvec{\mu }_D \right]$, a prior or process model $\left[ \varvec{\zeta }|{\varvec{C}}, \varvec{{\varSigma }}\right]$, and parameter priors $\left[ \varvec{\mu }_D \right]$, $\left[ {\varvec{C}}\right]$, $\left[ \varvec{{\varSigma }}\right]$, where $\varvec{\mu }_D$ contains data-model parameters and ${\varvec{C}}$ and $\varvec{{\varSigma }}$ represent the prior model parameters (Cressie and Wikle 2011; Sain et al. 2011; MacNab 2016a, b).

The MCAR and cMCAR constructions via compatible conditionals are tailor-made for MCMC implementations using Gibbs sampling and related methods (Besag et al. 1991). In Bayesian GLMM analysis, MCMC simulations are often implemented using related Metropolis–Hasting (MH) algorithms for posterior sampling of the fixed effects and the prior parameters, using Gibbs (or MH-within-Gibbs) samplers for posterior sampling of the random effects $\varvec{\zeta }$ or $\varvec{\eta }$ (Sain and Cressie 2007; Sain et al. 2011; Kim et al. 2001; Gelfand and Vounatsou 2003; Greco and Trivisano 2009; MacNab and Gustafson 2007; MacNab 2007, 2011, 2016a, b). In this section, some computational options are briefly outlined, mainly in the context of MCMC implementations in WinBUGS.

For a conditionally defined non-separable MCAR, we can program Gibbs algorithms in relation to the corresponding conditionals. For a linear coregionalization MCAR, the latent components often have simpler conditionals. Gibbs sampling for the latent components $\varvec{\eta }$ is often easier to program. Posterior samples of $\varvec{\zeta }$ may be calculated by letting $\varvec{\zeta }=\varvec{\eta }{\varvec{A}}^{\top }$; see Martinez-Beneito (2013) and MacNab (2016a, b) for technical details and WinBUGS examples. Implementation of a non-separable Mardia MCAR requires a square root factorization of $\varvec{{\varSigma }}=\varvec{{\varSigma }}^{1/2} \varvec{{\varSigma }}^{1/2}$, which can be a considerable computational burden, particularly for large values of p. On the other hand, the abovementioned MCMC methods for a Sain et al. MCAR and its coregionalization MCAR equivalent do not involve matrix factorization, which can be a significant computational advantage. Some WinBUGS examples for fitting the linear coregionalization MCARs are presented in the Supplementary Material for the paper.

For a cMCAR built from independent univariate CARs, the latent components in $\varvec{\eta }$ may be sampled from off-the-shelf algorithms or available syntaxes for the univariate CARs. Consider the cMpCAR build by independent pCARs, for instance, the WinBUGS syntax for pCAR may be used to sample the latent components in $\varvec{\eta }$; also see Martinez-Beneito (2013) for technical details and WinBUGS examples.

Implementation of SVCs build via $\hbox {vec}(\varvec{\zeta }) = {\varvec{H}}\hbox {vec}(\varvec{\eta })$ or $\hbox {vec}(\varvec{\zeta }^{\top } ) = {\varvec{H}}\hbox {vec}(\varvec{\eta }^{\top })$ typically involves factorization of a sparse spatial precision matrix or matrices. A SVC may be implemented by utilizing sparse matrix methods for the estimation of ${\varvec{T}}({\varvec{c}})$ or ${\varvec{T}}({\varvec{C}})$ via a spectral decomposition of ${\varvec{T}}({\varvec{c}})$ (or ${\varvec{T}}({\varvec{C}})$), the method presented in Sect. 4.3.1, or using a sparse Cholesky decomposition algorithm to increase computational efficiency (Rue and Held 2005; Furrer and Sain 2010; Sain et al. 2011; Pourahmadi 2013). This may be implemented in R or R-INLA (http://www.r-inla.org/), where fast algorithms for sparse matrix decomposition are available. For a SVC of the Type III decomposition, posterior sampling of the latent components $\varvec{\eta }\hbox {s}$ can be readily implemented for $\varvec{\eta }_{i \varvec{.}} \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varGamma }})$. For a SVC of the Type IV decomposition, Gibbs sampling for the latent components may be programmed with regard to the corresponding latent separable MCAR, formulated by univariate or multivariate conditionals. It is worth mentioning that if the ${\varvec{T}}({\varvec{c}})$ or ${\varvec{T}}({\varvec{C}})$ is a Cholesky decomposition of the associated CAR or MCAR precision matrix, the resulting MGMRF is not label-invariant with respect to the site-labeling.

8 Illustration via selected models

In this section, we briefly describe two applications of selected models in Bayesian hierarchical analysis of disease mapping data. We present results from the reanalysis of two data sets: Minnesota cancer mortality data for multivariate disease mapping (Jin et al. 2007; MacNab 2016a, b) and British Columbia adverse medical events data for spatiotemporal disease mapping (MacNab 2007). In both applications, the MGMRFs are used as prior models to facilitate flexible smoothing and to measure uncertainty. Bayesian prediction and inference are implemented in previously mentioned WinBUGS (WinBUGS examples are presented in the Supplementary Material for the paper).

For brevity, model comparisons are only briefly reported here using deviance information criterion (DIC) (Spiegelhalter et al. 2002): A smaller DIC indicates better balance of goodness-of-fit and complexity. The DIC is a commonly used model comparison tool used in Bayesian disease mapping (Jin et al. 2007; Greco and Trivisano 2009; Martinez-Beneito 2013; Lawson 2013). It is worth mentioning that other potentially useful model comparison tools, such as posterior predictive loss (Gelfand and Ghosh 1998), posterior predictive p value (Gelman et al. 1996), Gneiting and Raftery logarithmic score (2007), Watanabe Akaike information criterion (Watanabe 2010), and root mean squared error have also been used in the context of Bayesian disease mapping (Gelfand and Vounatsou 2003; MacNab 2011; Lawson 2013; MacNab 2016a; Martinez-Beneito et al. 2017; Ugarte et al. 2017).

8.1 Multivariate disease mapping

The Minnesota cancer mortality data that will be used in this application have previously been analyzed by various authors (for example, see Jin et al. and MacNab 2016a, b). The data set consists of observed and expected mortality counts, denoted $\{y_{ij}\}$ and $\{E_{ij}\}$ hereafter, for cancers of the esophagus ($j=1$), larynx ($j=2$), and lung ($j=3$), respectively, where $i=1, 2, . . . ,n$ are the labeling of $n=87$ counties in the Minnesota state of USA. We consider the Bayesian GLMM models presented in MacNab (2016a, b). Specifically, the data model for cancers of esophagus and lung are Poisson, $y_{ij} \sim \hbox {Poisson}(E_{ij} \hbox {exp}(b_{ij}))$, for $j=1,3$, and a zero-inflated Poisson (ZIP) data model is assumed for the cancer of larynx (due to an extremely rare disease and excessive zeros in the data), $y_{i2} \sim \hbox {Poisson}((1-\omega _i)E_{i2}\hbox {exp}(b_{i2})), \omega _i \sim \hbox {Bernoulli}(p)$; $\forall i,j$, $b_{ij} = b_{0j} + \zeta _{ij}$, $\hbox {E}(\zeta _{ij})=0$.

Table 1 DIC results for selected multivariate models

Fig. 1

Comparison of posterior relative risk median and standard deviation for indicated models with an asymmetric matrix ${\varvec{C}}$ (“cMpCAR(C)”) or a diagonal matrix ${\varvec{C}}$ (“cMpCAR(R)”): “cMpCAR(C) threefold” and “cMpCAR(R) threefold”—linear coregionalization recasts of threefold pCARs of the Type I decomposition; “cMpCAR(C) UC” and “cMpCAR(R) UC”—linear coregionalization models of the Type II decomposition

Fig. 2

Posterior relative risk predictions (median and standard deviation) for esophageal and lung cancers, based on (1) the M-model—blue circles, (2) the linear coregionalization cMpCAR(diag(${\varvec{C}}), \varvec{{\varSigma }}$)—red circles, (3) the twofold cMpCAR(${\varvec{C}}, \varvec{{\varSigma }}$)—red squares, and (4) the spatially varying coregionalization model $\hbox {SVC}_{\tiny \hbox {(I)}}$MpCAR(diag(${\varvec{C}}), \varvec{{\varSigma }}$)—red triangles, respectively. For the twofold cMpCAR, the matrix ${\varvec{C}}$ of spatial parameters is assumed to be symmetric and estimated based on a reparameterization via spectral decomposition. For the SVC, the spatial components are defined by variable-specific pCAR precision matrices and associated spectral decomposition (color figure online)

Overall, the various MCAR and coregionalization MCAR models led to comparable posterior relative risk predictions and inferences. Table 1 presents DIC results for selected models. Figure 1 additionally illustrates the similarities and differences among selected models, in terms of posterior relative risk medians and standard deviations. The differences observed are modest and are mostly observed among the posterior standard deviations for rare cancers such as esophageal, and laryngeal cancer in particular (the rarest cancer among the three). For lung cancer, the most common cancer among the three, the posterior risk medians and standard deviations are robust to prior choices. Overall, the coregionalization models have slightly smaller DICs compared with their counterparts of MCAR models. Models that allowed for asymmetric matrix ${\varvec{C}}$ performed slightly better than their counterparts that only allowed for diagonal matrix ${\varvec{C}}_{\tiny \hbox {diag}}$; this was observed through smaller DICs (see Table 2) and generally smaller posterior risk standard deviations. Similar results are also seen in recent publications (Jin et al. 2007; Greco and Trivisano 2009; Martinez-Beneito 2013; MacNab 2016a, b). The adaptive coregionalization models are comparable to, or having marginally smaller DICs than, their non-adaptive counterparts. Some differences were observed among MGMRFs formulated by univariate and multivariate conditionals, respectively: MCAR and cMCAR models of the Type II decomposition slightly outperformed their counterparts of the Type I decomposition.

Table 1 also presents results of illustrative examples of spatially varying coregionalization models. Compared to MCARs and cMCARs, SVCs imposed notably less smoothing. We also observe smaller deviances and comparable DICs for the MpCARs, but notably larger DICs for the MbCARs. The DIC and posterior risk predictions do not suggest the need for $\hbox {SVC}_{\tiny \hbox {(II)}} \hbox {s}$ over their (simpler) $\hbox {SVC}_{\tiny \hbox {(I)}}$ counterparts. Table 1 additionally presents the DIC results for the iCAR and LCAR models and their spatially varying coregionalization reconstructions. The latter are shown to have modestly lower values of deviance and DIC.

Figure 2 illustrates the similarities and differences between the models by the different types of decomposition. The results are derived from the estimated bivariate models using the data for esophageal and lung cancers. The estimated M-model and the cMpCAR, both are models of the Type II decomposition, are shown to impose comparable risk smoothing and lead to similar posterior relative risk predictions for esophageal and lung cancers. The twofold model of the Type I decomposition and the spatially varying coregionalization model of the Type III decomposition lead to notably less risk smoothing, with larger posterior risk standard deviations.

Table 2 lists the posterior estimates of the conditional spatial and cross-spatial correlation functions in ${\varvec{B}}$ (without the scaling factors), for the two order-free cMpCARs defined by linear combination of independent pCARs and for the $\hbox {cMpCAR}_{\tiny \hbox {Sain et al. UC}}$ (${\varvec{D}}_m={\varvec{D}}_w$) with a symmetric matrix ${\varvec{C}}$ of spatial parameters. These three models all resulted in positive posterior estimates of the spatial autocorrelation functions ${\varvec{B}}^*_{jj} \hbox {s}$; however, modest differences were observed among them. With high posterior uncertainties, the M-model and the cMpCAR with a symmetric coregionalization coefficients matrix ${\varvec{A}}$ both resulted in negative posterior estimates for the cross-spatial correlations. The posterior estimates of the cross-spatial dependence parameters in $\hbox {cMpCAR}_{\tiny \hbox {Sain et al. UC}}$ were positive, also with high posterior uncertainties.

Table 2 Posterior estimates of the symmetric matrix ${\varvec{B}}^*$, where $B^*_{jj} = { B_{jj} \over {\varGamma }_{jj}}$ and $B^*_{jl} ={ B_{jl} \over \sqrt{{\varGamma }_{jj} {\varGamma }_{ll}}}$ for the indicated MCAR models that postulate symmetric cross-spatial dependencies

8.2 Spatiotemporal disease mapping

The British Columbia adverse medical events (AME) data set was previously analyzed in the context of Bayesian spatiotemporal disease mapping (MacNab 2007). The data comprise the observed AME counts and the associated at risk population estimates for boys aged 1-19, denoted $\{y_{it}\}$ and $\{n_{it}\}$ hereafter, where $i=1,\ldots , n$ and $t=1,2,\ldots ,T$ for data spanning ten ($T=10$) years (1991–2000) and for $n=84$ local health areas in British Columbia, Canada; also see MacNab (2007) for further details about the data and related models and results.

As illustrative examples, we present results from two classes of spatiotemporal models. The first class of models are the B-spline models (with four inner-knots) presented in MacNab (2007): $y_{it} \sim \hbox {Poisson}(n_{it} \hbox {exp}(m_t+b_{it}))$,

$$\begin{aligned} m_t = a_0 +S_0(t), \,\, b_{it} = \zeta _{i0}+RS_i (t),\,\, S_0(t)=\sum _{k=1}^7a_k B_k(t), \,\, RS_i (t)=\sum _{k=1}^7\zeta _{ik} B_k(t) \end{aligned}$$

where $\{B_k(t)\}_{k=1}^7$ are the 4-knot B-spline basis functions without the intercept. We explored selected priors for the zero-mean random coefficients $\hbox {vec} (\varvec{\zeta }^{\top })=(\zeta _{10}, \zeta _{11},\ldots , \zeta _{17},\ldots ,\zeta _{n0}, \zeta _{n1}, \ldots , \zeta _{n7})^{\top }$. Here, we briefly report on results from three fitted models, each with a different prior for the random effects $\hbox {vec} (\varvec{\zeta }^{\top })$: (1) 8-variate M-based MpCAR (denoted M-model), (2) cMpCAR 8-fold diag(${\varvec{c}}$), and (3) non-spatial multivariate Gaussian (MIID) model, i.e., $\varvec{{\varOmega }}_{\hbox {vec}~(\varvec{\zeta }^{\top })} = {\varvec{I}}_n \otimes \varvec{{\varGamma }}$, as presented in MacNab and Gustafson (2007). The B-spline model with a MGMRF prior on $\hbox {vec} (\varvec{\zeta }^{\top })$ is a fully identified spatiotemporal model in which the random effects, $b_{it} \hbox {s}$, capture complex spatiotemporal risk interactions; they also model spatially varying nonlinear temporal risk patterns across geographical areas through spatially varying B-splines.

The second class of models are spatiotemporal models with locally adaptive $\hbox {SVC}_{\tiny \hbox {(I)}}$ prior(s) (denoted $\hbox {adaptSVC}_{\tiny \hbox {(I)}}$ hereafter) in which the non-spatial temporal relations are modeled as order-1 random walk (pRW1), order-2 random walk (RW2), and linear trends, respectively; see Rue and Held (2005) for additional details for RW1 and RW2 formulations. Once again, Poisson likelihood is assumed. Here, we report on three relatively simple adaptive $\hbox {SVC}_{\tiny \hbox {(I)}}$ models for the random effects $b_{it}=u_{i1}+\beta _{it}$, where, for the linear trend model, $\beta _{it} = {\varvec{u}}_{i2} t$, or, for the RW1 and RW2 process models,

$$\begin{aligned} \hbox {vec}(\varvec{\beta }) = ({\varvec{I}}_T \otimes \hbox {diag}(\varvec{\sigma }) ) \hbox {vec}(\varvec{\eta }), \end{aligned}$$

$\varvec{\sigma }=(\sigma _1, \ldots , \sigma _n)$, $\varvec{\eta }$ is a N by T matrix of latent components, $\hbox {vec}(\varvec{\eta }) \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varGamma }}\otimes {\varvec{I}}_n)$, $\varvec{{\varGamma }}$ is a T by T RW1($\lambda$) or RW2($\lambda =1$) precision matrix, where $\lambda$ is a temporal partial correlation parameter. In the three models, we let the $m_t \hbox {s}$ represent fixed effects, $\hbox {log} (\sigma _i)=\alpha +u_{i2}$, $\alpha$ is a fixed intercept; and $\hbox {vec}({\varvec{u}}) \sim \hbox {2-fold cMCAR}(\hbox {diag}({\varvec{c}}), \rho ^c)$ is a multivariate CAR prior used to characterize simple spatiotemporal interactions. The spatiotemporal model with pRW1 temporal characterization provides an example of an adaptive $\hbox {SVC}_{\tiny \hbox {(I)}}$ that gives rise to a non-singular MGRMF, whereas the spatiotemporal model assuming RW2 temporal processes leads to a singular MGMRF. Here, the identification issue related to the singular MGMRFs is handled by the commonly known sum-to-zero constraint.

Table 3 DIC results for selected spatiotemporal models

Among the three B-spline models, the M-model seems to have enabled most appropriate amount of smoothing, resulting in the lowest deviance and DIC (see Table 3). The 8-fold CAR, on the other hand, appears to have imposed the least favorable smoothing and exhibited the highest deviance and DIC.

Of the three locally adaptive coregionalization models, the model with proper RW1($\lambda$) for the latent temporal components at co-locations, denoted “$\hbox {adaptSVC}_{\tiny \hbox {(I)}}$ pRW(1) $\hbox {cMpCAR}_{\tiny \hbox {2-fold}}$ diag(${\varvec{c}}$)”, resulted in the lowest deviance and DIC. These results were also the best among all six spatiotemporal models. The model with RW2, denoted “$\hbox {adaptSVC}_{\tiny \hbox {(I)}}$ RW(2) $\hbox {cMpCAR}_{\tiny \hbox {2-fold}}$ diag(${\varvec{c}}$)”, and the model with linear trends, denoted “$\hbox {adaptSVC}_{\tiny \hbox {(I)}}$ linear trend $\hbox {cMpCAR}_{\tiny \hbox {2-fold}}$ diag(${\varvec{c}}$)”, were quite comparable: both imposed considerable smoothing. Consistent results are also observed among the posterior log relative risk predictions, as seen in Fig. 3.

Fig. 3

Posterior relative risk (RR) plots for selected local health areas, based on indicated models: dashed (red) line—crude RRs, solid (black) line: posterior median, with 2.5- and 97.5-percentiles (color figure online)

9 Discussion

In this paper, we provided a survey and synthesis of the main proposals for formulating conditionally specified MGMRFs. We presented a general coregionalization framework that connects and unifies the various lines of model development.

The Besag (1974), Mardia (1988) and Sain et al. (2011) frameworks provide the essential tools for formulations of a wide range of coregionalization MGMRFs, with extensions to locally adaptive models. The coregionalization framework presented herein contains the majority of the MGMRFs proposed in the literature to date. Its inclusiveness and computational advantages, highlighted in the presented paper, should offer a broad range of models and tools for a wide range of applications, including but not limited to those outlined in the introduction.

The coregionalization framework presented herein still has considerable potential for further extension and new model development, with potential connections to non-stationary or stationary multivariate Gaussian processes commonly discussed in the geostatistics literature (Gelfand et al. 2004; Lindgren et al. 2011).

In what follows, we present an organized summary discussion on insights gained from this study. We also discuss challenges and opportunities concerning model construction and parameterization, interpretation, and statistical computation.

9.1 The Sain et al. and Mardia frameworks

The Sain et al. framework provides tools for flexible characterizations of conditional spatial and cross-spatial dependencies. The MGMRFs discussed herein postulate and control for conditional cross-spatial dependencies via a full matrix of spatial dependence parameters. This family of MGMRFs may be conceptualized and defined over variable-specific lattice-neighbor schemes or over different lattices. The spatial and non-spatial dependence parameters in the Sain et al. MGMRFs discussed herein are partial correlation parameters of non-separable partial correlation matrices. These spatial and non-spatial partial correlation parameters are constrained together, due to complex requirement that the MGMRF precision (covariance) matrix must be positive definite.

The Sain et al. MGMRFs discussed herein use a full matrix of spatial dependence parameters to postulate and control for conditional spatial and cross-spatial dependencies. The diagonal dominance constraint for enforcing the positivity condition, and the currently available solutions for shrinkage estimation of a matrix C of spatial parameters, may reduce the intended flexibility of a proposed model.

The Mardia framework may have limited flexibility for intuitive characterizations of the conditional cross-spatial dependencies. However, it may enable the formulations of flexible and intuitively appealing MGMRFs with spatially smoothing covariance and cross-covariance functions, including asymmetric cross-covariance functions. With the arrival of linear and spatially varying coregionalization approaches to formulating MGMRFs, the Mardia framework offers an option to formulating linear coregionalization models with correlated latent components modeled by multivariate conditionals.

9.2 The broadened coregionalization framework

The linear and spatially varying coregionalization approaches to the formulations of multivariate spatial or spatiotemporal models presented herein offers a unified framework for a systematic development and interpretation of the major proposals of MGMRFs presented in the literature, with related computational methods for statistical estimation and inference. In spite the fact that the conditional formulation of (M)GMRFs “is marred by a number of disadvantages” (Besag 1974), one important message of the present study is that the Besag (1974), Mardia (1988) and Sain et al. (2011) frameworks provide the theoretical foundation, essential building blocks, and intuitively appealing motivations for building MGMRFs for finite lattice systems and within the coregionalization framework. It is worth noting that Besag (1974) and Mardia (1988) also discuss formulations of non-stationary and stationary MGMRFs (i.e., Gaussian processes) on infinite lattice. The (M)GMRF proposals presented in recent literature, and in the present paper, are specific (M)GMRF constructions for data observed over systems of finite lattice.

Coregionalization is a powerful tool for multivariate generalizations of univariate spatial models, dimension reduction, and multivariate statistics applications. Spatially varying coregionalization enables us to formulate p-variate GMRFs that allow asymmetric cross-covariance functions to be parameterized via a p-vector of spatial parameters. These spatially varying coregionalization MGMRFs have intuitive interpretations for their coregionalization covariance and cross-covariance matrix functions and may be computationally more manageable in detailing with large-n and/or large-p problems. Linear coregionalization models built using independent latent GMRFs, as well as their counterparts of spatially varying coregionalization models, have broad scope for coregionalization MGMRF formulation over variable-specific lattice-neighbor schemes or over different lattices.

9.3 MGMRF models for prediction, smoothing, and explanation: disease mapping and ecological regression

For multivariate disease mapping and spatial smoothing, the various approaches to formulating MGMRFs and linear coregionalization MGMRFs seem to perform comparably as smoothers. While quite preliminary, some consistent results have emerged from this study and from recent literature. Specifically, in our illustrative examples, the more flexible models, those that accommodate asymmetric cross-spatial dependencies, only slightly outperformed their simpler counterparts, which allowed for symmetric cross-spatial dependencies to be modeled by a diagonal matrix ${\varvec{C}}$. Similar results can be seen in recent publications (Jin et al. 2007; Greco and Trivisano 2009; Martinez-Beneito 2013; MacNab 2016a, b). Adaptive cMpCARs are seen to be comparable to, or to marginally outperform, their non-adaptive counterparts. Similar results were also seen in MacNab et al. (2006) and Congdon (2008b). The Mardia MGMRFs and their coregionalization model counterparts are shown to perform slightly better than their Sain et al. MGMRF counterparts.

The spatiotemporal models presented herein provide brief illustrations for the range of potentially useful models for analyzing spatiotemporal data. The spatially varying coregionalization MGMRFs may offer considerable options for flexible multivariate spatial and spatiotemoral smoothing. For example, spatially varying coregionalization constructions parameterized by component-specific spatial parameters ${\varvec{c}}$, with the latent components characterizing the variable or time domain $\hbox {vec}(\varvec{\zeta }) \sim \hbox {MVN}({\varvec{0}}, \varvec{{\varGamma }}\otimes {\varvec{I}}_n)$, may offer considerable model flexibility and options that are both conceptually plausible and computationally manageable.

Coregionalization models with locally adaptive spatial and/or scale parameters offer considerable flexibility for SVC constructions, from formulation of simpler models, say, the spatially varying adaptive coregionalization models illustrated in the paper, to those of more complex ones, say, more elaborate spatially adaptive $\hbox {SVC}_{\tiny \hbox {(I)}} \hbox {s}$ or $\hbox {SVC}_{\tiny \hbox {(II)}} \hbox {s}$. Locally adaptive models might be considered for data with complex multivariate spatial or spatiotemporal dependencies and heterogeneities, particularly in situations where explanatory covariates are available to model locally structured heterogeneity. We also anticipate potential applications of adaptive models in the context of joint mean-covariance (or inverse covariance) modeling (Pourahmadi 1999, 2000, 2007) of multivariate spatial and spatiotemporal data with explanatory covariates.

In summary, the basic linear coregionalization framework offers a broad range of model options for disease mapping and spatial regression. Spatially varying coregionalization also offers tools for reconstruction of CAR or MCAR for orthogonal random effects in GLMM disease mapping and spatial regressions.

9.4 Challenges and opportunities

Due, in part, to the highly restrictive constraints placed on the multivariate Gaussian precision matrices to enforce symmetry and positivity conditions, MGMRFs parameterized by the matrix ${\varvec{C}}$ of spatial parameters may have limited applications in modeling and inferring complex conditional spatial and cross-spatial dependencies, including (a)symmetric cross-spatial dependencies. The currently available approaches to shrinkage estimation of MGMRFs may also place limits on the utility of the MGMRFs. MGMRFs formulated on a “hardwired” lattice-neighborhood structure, with an undirected graph of connected edges representing Markovian dependencies in terms of conditional correlations and local interactions, may also have limited flexibility to model complex multivariate and multi-dimensional spatial dependencies and relations. This could be one important reason that high posterior uncertainties about conditional spatial cross-correlation parameters are suggested in the present study and in recent literature (Jin et al. 2007; Greco and Trivisano 2009; MacNab 2016a, b). Locally adaptive models that allow for more flexible spatial correlation structures, such as randomly connected edges (Brezger et al. 2007; Lee et al. 2014), may offer alternative and more flexible parameterizations and representations of multivariate spatial dependencies, local interactions, discontinuities, and asymmetry.

While the goal for flexible models is often to facilitate characterization of complex multivariate spatial and non-spatial dependencies and associations, the implementation of these complex models is often constrained by computational complexity, particularly in cases where $p > 3$. The present solutions to the shrinkage estimations of the spatial parameter matrix ${\varvec{C}}$ and the covariance matrix $\varvec{{\varSigma }}$ or $\varvec{{\varSigma }}^{1/2}$ seem to be tentatively useful for small values of p, say $p \le 3$. As p increases, we could be dealing with an unmanageable number of prior parameters. Reducing ${\varvec{C}}$ to a diagonal matrix often simplifies computation significantly. Another way to reduce computational complexity is to place a structural assumption on the non-spatial covariance matrix $\varvec{{\varSigma }}$, as illustrated in the present paper for spatiotemporal models. Fast computation for sparse matrix decomposition and efficient tools for covariance estimation may enable wide use of the linear and spatially varying coregionalization models. Some computational solutions may be found in Barnard et al. (2000), Gelfand and Vounatsou (2003), Gelfand et al. (2004), Rue and Held (2005), Sain and Cressie (2007), Jin et al. (2007), Sain et al. (2011) and Pourahmadi (2011, 2013); implementation of these solutions through programming in R or extensions of R-INLA may also be fruitfully explored.