Functional Linear Regression for Partially Observed Functional Data

Abstract

In functional linear regression model, many methods have been proposed and studied to estimate the slope function while the functional predictor was observed in the entire domain. However, works on functional linear regression model with partially observed trajectories have received less attention. In this paper, to fill the literature gap we consider the scenario where individual functional predictor maybe observed only on part of the domain. Depending on whether measurement error is presented in functional predictors, two methods are developed, one is based on linear functionals of the observed part of the trajectory and the other one uses conditional principal component scores. We establish the asymptotic properties of the two proposed methods. Finite sample simulations are conducted to verify their performance. Diffusion tensor imaging (DTI) data from Alzheimer’s Disease Neuroimaging Initiative (ADNI) study is analyzed.

1 Introduction

With the advance in technology, it is increasingly common to encounter data that are functions or curves in nature (see Ramsay 2005). Functional linear regression models provide a framework for modeling the dynamic relationship between response and functional predictors, which was first introduced by Ramsay and Dalzell (1991). One of the primary goals for functional linear model (FLM) is to get an estimator of functional coefficient. And many procedures have been proposed to approximate functional coefficient, for example, functional principal component analysis (FPCA) based approaches (Cardot et al. 1999; Hall and Horowitz 2007; Yao et al. 2005b), spline-based approaches (Crambes et al. 2009; Marx and Eilers 1999)), wavelet-based approaches (Zhao et al. 2012; Wang et al. 2019)), and others. We refer to Morris (2015) and Reiss et al. (2017) for more informative and extensive reviews on such functional linear models.

Among the different based methods in functional data analysis, FPCA based approaches for capturing the information of covariates are popular (Hall et al. 2006; Che et al. 2017). In the setting where trajectories are observed on dense and regular grid on the entire domain, the existing works can be found in Besse and Ramsay (1986), Rice and Silverman (1991), Cardot et al. (1999), Shin (2009), Horváth and Kokoszka (2012), to name a few. Yao et al. (2005a) emphasize the case where the functional predictors are observed with irregularly sparse measurements which is often referred to as sparse functional data and proposes a nonparametric method to perform FPCA. For general review on FPCA, see Shang (2014). In this paper, we prefer to use FPCA method to get an estimator of the functional coefficient.

Sparse functional data addresses the case where each trajectory is observed at a small number of points that are distributed randomly on the domain which is different from the partially observed functional data (or incomplete or fragmentary functional data) which was first introduced in Liebl (2013). Partially observed functional data addresses each trajectory is observed at points that cover a subset of the domain in such a way that trajectories can be reasonably treated as fragments of curves (Delaigle and Hall 2016) that has great implication in applications, such as in biomedicine, economics (see Kraus 2015; Kneip and Liebl 2020). Considering the partially observed functional data can be treated as missing data for functional curves over the domain, two missing mechanisms are introduced in the existing works: one is missing completely at random (MCAR), that is, the missing data mechanism is independent from other stochastic components (Delaigle and Hall 2016; Goldberg et al. 2014); the other one is the missing mechanism in which depends on systematic strategies, such as missing parts of the trajectories only occur at the upper interval of the domain (see Liebl and Rameseder 2019). In the setting of MCAR, Delaigle and Hall (2016), Goldberg et al. (2014) and Kraus (2015) address the problem for recovering the missing parts of trajectories. Kraus (2015) and Kneip and Liebl (2020) model the functional principal (FPC) scores of an incomplete trajectory. In the scenario where missing data mechanism depends on systematic strategies, Liebl and Rameseder (2019) establishes estimators for the mean and the covariance function of the incomplete functional data via the fundamental theorem of calculus. To the best of our knowledge, no work exists focusing on estimating functional coefficient of FLM with partially observed trajectories.

In this paper, we address the problem of getting an estimator of functional coefficient for the case of partially observed functional data without and with measurement error. In the scenario that trajectories observed without measurement error, instead of deleting the incomplete trajectories, we get estimators of FPC scores for each incomplete trajectory by modeling it as linear functionals of the observed parts of that trajectory. In the setting where trajectories observed with measurement error, we use local linear smoother methods to estimate mean and covariance function of the functional predictor, followed by getting FPC scores via conditional expectation.

The contributions of this paper are as follows. First, we extend FLM approach to partially observed functional data without measurement error, which leads to an improved estimator for functional coefficient comparing with the one obtained through deleting the incomplete trajectories for given dataset. Second, we develop an estimate method for functional coefficient in FLM for incomplete trajectories with measurement error. We illustrate its usefulness by comparing with another two methods: one is based on integration method to get the FPC scores of the functional predictor instead of using conditional expectations; the other estimator is obtained by ignoring the measurement error of the trajectories in the dataset. Third, in both scenarios, we obtain the rate of convergence for the proposed estimators. Overall, the methodological and numerical developments in this paper can provide a practically useful way in analyzing FLM with partially observed functional data.

The rest of this paper is organized as follows. In Sect. 2, we introduce functional linear models. In Sect. 3.1, we develop an estimator for functional coefficient with incomplete trajectories observed without measurement error and establish theoretical properties for the proposed estimator. An estimator and theoretical properties in the scenario that incomplete trajectories observed with measurement error are introduced in Sect. 3.2. Section 4 illustrates the finite sample performance of our proposed estimators through simulation studies, followed by a real data analysis in Sect. 5. Discussion is presented in Sect. 6. Proofs of theorems are given in the Appendix.

2 Functional Linear Model

Consider a functional linear model, in which the scalar response Y _i is linearly related to the functional covariate X _i,

$$\displaystyle \begin{aligned} Y_i= \alpha + \int_{\mathcal{T}}\gamma(t)X_i(t)dt+\epsilon_i, {} \end{aligned}$$

(1)

where α is the intercept, $\{X_i(t): t\in \mathcal {T},i=1,\ldots ,n\}$ are the functional predictors, sampled from the stochastic process $\{X(t):t\in \mathcal {T}\}$ with mean function μ, domain $\mathcal {T}$ is bounded and closed, γ is the slope function to be estimated, 𝜖 _i are random errors satisfying E[𝜖 _i] = 0, $\text{E} [\epsilon _i^2]=\sigma ^2<\infty$. We can easily get an estimator of intercept once we get an estimator of γ. So we focus on estimating γ in the following (Hall and Horowitz 2007). Let 〈⋅, ⋅〉, ||⋅|| be the inner product and norm on $L^2(\mathcal {T})$, the set of all square integrable functions on $\mathcal {T}$, with $\langle f, g \rangle = \int _{\mathcal {T}}f(t)g(t)\text{dt}$, ∥f∥ = 〈f, f〉^1∕2 for any $f,g\in L^2(\mathcal {T})$.

We first recall the method FPCA in estimating the slope function for model (1) with the functional predictor X _i observed on the entire domain $\mathcal {T}$. For the stochastic process $X\in L^2(\mathcal {T})$, denote its mean function as μ: μ = E(X), and its covariance function as c _X(s, t): c _X(s, t) = cov(X(s), X(t)). Assume c _X is continuous on $\mathcal {T}\times \mathcal {T}$. The expression $c_X(s,t)=\sum _{j=1}^\infty \lambda _j\phi _j(s)\phi _j(t)$ exists by the Mercer Lemma (Riesz and Nagy 1955), where λ ₁ > λ ₂ > ⋯ > 0; ϕ ₁, ϕ ₂, ⋯ are the eigenvalue sequence and the continuous orthonormal eigenfunction sequence of the linear operator C _X: $(C_X\phi )(\cdot ) = \int _{\mathcal {T}} c_X(\cdot ,t)\phi (t)\text{dt}$, $\phi \in L^2(\mathcal {T})$, with the kernel c _X. On the other hand, by the Karhunen–Loève (K-L) expansion, one has $X_i(t)=\sum _{j=1}^\infty U_{ij}\phi _j(t),$ where the random variables U _ij = 〈X _i − μ, ϕ _j〉 are uncorrelated with $\text{E} [U_{ij}]=0,\,\text{E} [U_{ij}^2]=\lambda _j$, and $\gamma (t)=\sum _{j=1}^\infty \gamma _j\phi _j(t)$ with γ _j = 〈γ, ϕ _j〉.

The full model (1) is then equivalent to $Y_i - \text{E}Y_i= \sum _{j=1}^\infty \gamma _j U_{ij}+\epsilon _i$ based on K-L expansion, which can be approximated by $\sum _{j=1}^m\gamma _j {U}_{ij}+\epsilon _i$ by using the first m terms. To simplify notations, we assume that {Y _i, i = 1, ⋯ , n} are centered. Let Y = (Y ₁, ⋯ , Y _n)^T, γ = (γ ₁, ⋯ , γ _m)^T, $\hat {\mu }$ be an estimator of μ, $\{\hat {\lambda }_j\}$ and $\{\hat {\phi }_j\}$ be estimators of the sequence {λ _j} and {ϕ _j} with $\hat {\lambda }_1>\hat {\lambda }_2>\cdots >0$. The least square estimator $\hat {\mathbf {{\gamma }}}$ is then given as

$$\displaystyle \begin{aligned} \hat{\mathbf{{\gamma}}}=\left({\hat{\mathbf U}_m}^T\hat{\mathbf U}_m\right)^{-1}\hat{\mathbf U}_m \mathbf Y,{} \end{aligned} $$

(2)

provided that $({\hat {\mathbf U}_m}^T\hat {\mathbf U}_m)^{-1}$ exists with $\hat {U}_{ij}=\langle X_i - \hat {\mu },\hat {\phi }_j\rangle$, $\hat {\mathbf U}_m=(\hat {U}_{ij})_{\substack {i=1,\cdots ,n;\\ j=1,\cdots ,m}}$. Moreover, for the estimator $\hat {\gamma }_j, j=1,\cdots ,m$, it has the equivalent form as

$$\displaystyle \begin{aligned} \hat{\gamma}_j = \hat{\lambda}^{-1}_{j}\left\langle n^{-1}\sum_{i=1}^{n} (Y_i-\bar{Y}_0)(X_i-\hat{\mu}), \hat{\phi}_{j}\right\rangle. \end{aligned}$$

Consequently, an estimator of γ is given by

$$\displaystyle \begin{aligned} \hat \gamma(t)=\sum_{j=1}^m \hat{\gamma}_j\hat{\phi}_{j}(t). {} \end{aligned}$$

(3)

The number m of included eigenfunctions is chosen by fraction of variance explained criterion in practice (James et al. 2000): $m=\text{min}\{k:\sum _{l=1}^{k}\hat {\lambda }_l/\sum _{l=1}^{n}\hat {\lambda }_i\geq R \}$, with a given threshold R. For the asymptotic analysis, we assume m depends on sample size n such that m →∞ as n →∞.

3 Estimation Methods

The above analysis is based on the assumption the functional predictor is observed on the entire domain. We now consider the scenario that the predictor X _i, i = 1, ⋯ , n may be available only on parts of $\mathcal {T}$. We first give some notations and then make further analysis. Let X ₁, ⋯ , X _n be an independent and identically distributed samples from the random function X. We denote the observed and missing parts of X _i by O _i and M _i with $O_i\cup M_i=\mathcal {T}$. Let $O_i=[L_i,R_i]\subseteq \mathcal {T}$, and assume that it is a random subinterval independent of X _i with R _i − L _i > 0 almost surely. The observed data for ith functional predictor is then given as X _i(t), t ∈ O _i, i = 1, ⋯ , n, denoted by $X_{iO_i}$. In this section, our objective interest is to develop an estimation method for model (1) with partially observed functional observations without and with measurement error, respectively. And in these scenarios, our objective is to get estimators of the functional principal component scores {U _ij} and the eigenfunctions {ϕ _j} as indicated in formulas (2) and (3). Depending on whether measurement error is presented in partially observed functional curves, two methods are developed: one is established by applying linear functionals of the observed parts of that trajectory, while the other one is based on principal component analysis through conditional expectation.

3.1 Partially Observed Functional Data Without Measurement Error

In the scenario that functional curves are partially observed on the domain without measurement error, to get an estimator of γ in model (1), we need to get estimators of U _ij and ϕ _j pertaining to this case. An estimator of U _ij is obtained based on the linear functional of the observed part $X_{iO_i}$, and an estimator of ϕ _j is obtained by giving estimators of mean and covariance function of X. The steps are given here.

Step 1::

Estimate the mean μ and the covariance function c _X by sample mean and sample covariance.

Step 2::

Estimate eigenvalues {λ _j} and eigenfunctions {ϕ _j} by $\int _{\mathcal {T}} \hat {c}_X(s,t)\hat {\phi }_j(s)$ $ds = \hat {\lambda }_j\hat {\phi }_j(t).$

Step 3::

Estimate principal component scores $U_{ij}=U_{ijO_i}+U_{ijM_i}$ with $\hat U_{ijO_i} = \langle X_{iO_i} - \hat {\mu }_{O_i},\hat {\phi }_{jO_i}\rangle$, and estimate $U_{ijM_i}$ by modeling it as linear functionals of $X_{iO_i}$ given as $\hat U_{ijM_i} = \langle \hat {\xi }_{ijM_i},X_{iO_i} - {\hat {\mu }}_{O_i}\rangle$.

Step 4::

Estimate γ based on formulas (2) and (3) for $X_{iO_i}$ observed without measurement error.

We first address the problem of getting estimators of μ and c _X, denoted as $\hat {\mu }^{\text{NME}}$ and $\hat {c}^{\text{NME}}_X$, respectively, followed by establishing estimators of U _ij and eigenfunctions ϕ _j which are denoted as $\hat {U}^{\text{NME}}_{ij}$ and $\hat {\phi }^{\text{NME}}_j$. For simplicity of presentation, we suppress the notation on “NME” in this subsection unless otherwise stated.

Let $O_i(t) = \text{I}_{O_i}(t)$ with indicator function $\text{I}_{O_i}(t)$ being 1 if t ∈ O _i, and 0 otherwise, and let W _i(s, t) = O _i(s)O _i(t). The estimators of the mean function μ and the covariance function c _X of X obtained from the observed points s, t of X _i, are given by,

$$\displaystyle \begin{aligned} \hat{\mu}(t) = \frac{1}{\sum_{i=1}^n O_i(t)} \sum_{i=1}^n O_i(t) X_i(t), {} \end{aligned} $$

(4)

$$\displaystyle \begin{aligned} \hat{c}_X(s,t) = \frac{1}{\sum_{i=1}^{n} W_i(s,t)}\sum_{i=1}^{n} W_i(s,t) (X_i(s) - \hat{\mu}(s))(X_i(t) - \hat{\mu}(t)). {} \end{aligned} $$

(5)

Therefore, we get the estimators $\{\hat {\lambda }_j\}$, $\{\hat {\phi }_j\}$ related to {λ _j} and {ϕ _j} from $\hat {c}_X$ associated with the covariance operator $\hat {C}_X$.

We could not get estimators $\hat {U}_{ij}$ of FPC scores {U _ij} of X _i directly from its definition if $O_i\ne \mathcal {T}$. To bridge the gap, U _ij is decomposed into two parts:

$$\displaystyle \begin{aligned} U_{ij}=\langle X_{iO_i} - \mu_{O_i},\phi_{jO_i}\rangle+\langle X_{iM_i} - \mu_{M_i},\phi_{jM_i}\rangle=U_{ijO_i}+U_{ijM_i}, {} \end{aligned} $$

(6)

where $\mu _{O_i}$ and $\phi _{jO_i}$ denote the restriction of μ and the eigenfunction ϕ _j on O _i, respectively, and the definitions of $\mu _{M_i}$, $\phi _{jM_i}$ are similar. The estimator $\hat {U}_{ijO_i}$ of $U_{ijO_i}$ can be estimated directly from the observed part $X_{iO_i}$ and the estimator $\hat \phi _j$, given as $\hat {U}_{ijO_i} = \langle X_{iO_i}-\hat {\mu }_{iO_i}, \hat \phi _{jO_i}\rangle$. For the term $U_{ijM_i}$, we consider using the linear functional form $\langle \xi _{ijM_i},X_{iO_i} - {\mu }_{O_i}\rangle$ of the observed part $X_{iO_i}$ to estimate it which is also considered in Kraus (2015), that is,

$$\displaystyle \begin{aligned} \hat{\xi}_{ijM_i} = \underset{\xi_{ijM_i}\in L^2}{\text{argmin}}\,\, n^{-1}\sum_{i=1}^{n} (\hat{U}_{ijM_i} - \langle \xi_{ijM_i},X_{iO_i}-\hat{\mu}_{iO_i}\rangle)^2 \end{aligned}$$

with ${\hat {U}}_{ijM_i} = \langle X_{iM_i}-{\hat {\mu }}_{M_i}, {\hat {\phi }}_{jM_i}\rangle$. The estimator ${\hat {\xi }}_{ijM_i}$ has the explicit form: $\hat {\xi }_{ijM_i}={{\hat C}_{O_iO_i}}^{-1}{\hat C}_{O_iM_i}{\hat {\phi }}_{jM_i}$, where $\hat C_{O_iO_i}$, $\hat C_{O_iM_i}$ are the empirical covariance operator for $C_{O_iO_i}$, $C_{O_iM_i}$ with the kernel being the covariance function $\hat c_X$ of X _i restricted to O _i × O _i and O _i × M _i, respectively. To obtain a stable solution, we adopt ridge regularization, given by

$$\displaystyle \begin{aligned} &\hat{\xi}_{ijM_i}^{(\rho)}={(\hat C^{(\rho)}_{O_iO_i})}^{-1}\hat C_{O_iM_i}\hat{\phi}_{jM_i},\\ &\hat U^{(\rho)}_{ijM_i}=\langle\hat{\xi}_{ijM_i}^{(\rho)},X_{iO_i} - \hat{\mu}_{iO_i}\rangle, i=1,\cdots,n,\,j=1,\cdots,m, {} \end{aligned} $$

(7)

where $\hat {C}_{O_iO_i}^{(\rho )}=\hat {C}_{O_iO_i}+\rho \mathcal {F}_{O_i}$, $\mathcal {F}_{O_i}$ is an identity operator defined on L ²(O _i), ρ is a ridge parameter; see Kraus (2015) for further details. Let $\hat {U}^{\text{NME}}_{ij} = \hat {U}_{ijO_i} + \hat U^{(\rho )}_{ijM_i}$. The estimator $\hat {\gamma }^{\text{NME}}$ of γ using all of the information of the dataset is then obtained through replacing $\hat {U}_{ij}$ in (2) with $\hat {U}^{\text{NME}}_{ij}$,

$$\displaystyle \begin{aligned} \hat {\gamma}^{\mathrm{NME}}(t)=\sum_{j=1}^m \hat{\gamma}_j\hat{\phi}_{j}(t). {} \end{aligned} $$

(8)

To facilitate our theoretical analysis, we first impose some assumptions on observation points for partially observed functional curves, indicating the observation points asymptotically provide enough information in individual or pairwise crossover.

• (A1) $\text{There exists} ~\delta _1>0 ~\text{s.t.} \mathop {\text{sup}} \limits _{t\in [0,1]}\text{P}\{n^{-1}\sum _{i=1}^n \text{I}_{O_i}(t)\leq \delta _1\}=O(n^{-2})$.

• (A2) $\text{There exists} ~\delta _2>0 ~\text{s.t.} \mathop {\text{sup}} \limits _{s,t\in [0,1]^2}\text{P}\{n^{-1}\sum _{i=1}^nW_i(s,t)\leq \delta _2\}=O(n^{-2}).$

Moreover, we also introduce some regularity conditions necessary to derive theoretical properties for the estimate $\hat {\gamma }^{\text{NME}}$.

• (A3) E||X − μ||⁴ < ∞.

• (A4) nm ⁻¹ →∞, $n/(\sum _{j=1}^{m}\delta ^{-2}_j)\rightarrow \infty$ with δ _j = min_j≥1{λ _j − λ _j+1, λ _j−1 − λ _j} and $n\lambda ^2_m\rightarrow \infty$ as m →∞.

• (A5) The ridge parameter ρ satisfies ρ → 0, nρ ³ → 0, nm ⁻¹ ρ ² →∞.

• (A6) $\sum _{k=1}^{\infty }[\text{E}[YU_k]]^2/\lambda _k^2<\infty$.

• (A7) $\sum _{j=1}^\infty \sum _{k=1}^\infty \frac {r_{M_iO_ijk}^2}{\lambda _{O_iO_ik}^2}<\infty ,$ with $r_{M_iO_ijk}=\text{cov}(\langle X_{M_i}-\mu _{M_i},\phi _{M_iM_ij}\rangle , \langle X_{M_i}-\mu _{M_i}, \phi _{O_iO_ik}\rangle )$.

Assumption (A3) is a common condition in the analysis of functional model by using the method of FPCA to guarantee the random functions have finite fourth moment (see Cardot et al. 1999). Note that if the eigenvalues {λ _j} are exponentially or geometrically decreasing, the assumption (A4) holds. The same kind of conditions are also introduced in Cardot et al. (1999). Assumption (A5) is used to control the size of ridge effect. To define the convergence of the right hand of the formula $\gamma (s)=\sum _{k=1}^{\infty }(\text{E}[YU_k]/\lambda _k) \phi _k(s)$, in the L ² sense, assumption (A6) is required that is similar to the condition (A1) in Yao et al. (2005b). Assumption (A7) is used to make the solution $\hat {\xi }_{ijM_i}$ valid which is commonplace in the theory of inverse problems as Picard condition (see Hansen 1990).

Let $\theta _n=\sum _{k=m}^{\infty }[\text{E}[YU_k]]^2/\lambda _k^2$. Then assumption (A6) indicates that θ _n → 0. Denote $\upsilon = \sum _{j=1}^{m}V_{ij}$ with $V_{ij}= \langle \phi _{jM_i}, (C_{M_iM_i}- C_{M_iO_i}C^{-1}_{O_iO_i}C_{O_iM_i})\phi _{jM_i}\rangle .$ Based on the above assumptions, Theorem 1 gives the converge rate for the estimator $\hat {\gamma }^{\text{NME}}$ in the L ² sense.

Theorem 1

Suppose that (A1)–(A7) are satisfied. Then

$$\displaystyle \begin{aligned} \|\hat{\gamma}^{NME}-\gamma\|{}^2 = O_p(n^{-1}m\rho^{-2} + \iota_n + \theta_n + \upsilon), \end{aligned} $$

with $\iota _n = n^{-1}\sum _{j=1}^{m} \delta ^{-2}_j$.

Theorem 1 indicates that the approximation error rate of $\hat {\gamma }^{\text{NME}}$ for γ is controlled by four terms. The first term depends on sample size n, tuning parameter m, ridge parameter ρ, which is of the higher order than the one given in Hall and Horowitz (2007) that is mainly due to functional curves observed on the part of the domain. The second term is related to the spacings between adjacent eigenvalues, and its effect on convergence rate of γ is also emphasized in Hall and Horowitz (2007). The third term is related to the convergence of γ in L ² sense, which is also shown in Yao et al. (2005b) to get approximation error rate for functional coefficient. The fourth term is introduced by approximating $U_{ijM_i}$ with $\tilde {U}_{ijM_i}$.

Note that in practice, the ridge parameter ρ included in the regularized estimation of the jth score of the ith functional observation is chosen by generalized cross-validation based on the set of samples observed on the entire domain (see Kraus 2015).

3.2 Partially Observed Functional Data with Measurement Error

In this subsection, we construct an estimator for the slope function γ for partially observed trajectories with measurement error. We suppose the functional observations are:

$$\displaystyle \begin{aligned} Z_{il} = X_i(t_{il})+\varepsilon_{il},\quad t_{il}\in O_i, i=1,\cdots,n, l=1,\cdots N_i, {} \end{aligned} $$

(9)

where ε _il is independent from all the other variables X _j, j ≠ i, with E(ε _il) = 0, $\text{var}(\varepsilon _{il})=\sigma ^2_X$.

To get an estimator of γ in (1) in the scenario that trajectories may be observed on parts of the domain with measurement error (WME), we need give estimators of FPC scores and eigenstructure pertaining to this case. Estimator of eigenstructure is established after using local linear smoothers to get estimators of mean and covariance function of X. We obtain estimators of FPC scores by using approach of principal component analysis via conditional expectation. The steps are given here.

Step 1::

Estimate the mean and covariance functions by local linear smoothers.

Step 2::

Estimate eigenvalues {λ _j} and eigenfunctions {ϕ _j} by $\int _{\mathcal {T}} \hat {c}^{\text{WME}}_X(s,t)$ $\hat {\phi }^{\text{WME}}_j(s) ds = \hat {\lambda }^{\text{WME}}_j{\hat {\phi }}^{\text{WME}}_j(t)$.

Step 3::

Estimate FPC scores {U _ij} by principal component analysis via conditional expectation (PACE): $\tilde {U}_{ij} = \text{E}[U_{ij}|{\mathbf {Z}}_i]$.

Step 4::

Based on obtained estimators $\hat {\tilde {U}}_{ij}$ and $\hat {\phi }^{\text{WME}}_j$, we get estimator γ ^WME for $X_{iO_i}$ observed with measurement error.

We first calculate estimators for the mean and the covariance function of X in the scenario (9), denoted as $\hat {\mu }^{\text{WME}}$ and $\hat {c}^{\text{WME}}_X$, that are required to derive estimators for the FPC scores $U_{ij} = \int (X_i(t)-\mu (t))\phi _j(t)\text{dt}$. For simplicity of presentation, we suppress notation on “WME” unless otherwise stated in this subsection.

Let K(⋅) be a nonnegative univariate kernel function that is assumed to be a symmetric probability density function (pdf) with compact support supp(K) = [−1, 1], and h _μ, h _c be the bandwidths for obtaining estimators of μ, c _X. Assume that the second derivatives of μ, c _X on $\mathcal {T}$, $\mathcal {T}^2$, respectively, exist. We use local linear smoothers for the mean function μ (Yao et al. 2005a,b; Kneip and Liebl 2020) defined as $\hat {\mu }(t) = \hat {\beta }_0$, where

$$\displaystyle \begin{aligned} (\hat{\beta}_0,\hat{\beta}_1) = \underset{\beta_0,\beta_1}{\text{argmin}}\sum_{i=1}^{n}\sum_{l=1}^{N_i}K\left(\frac{t_{il}-t}{h_{\mu}}\right)[Z_{il}-\beta_0-\beta_1(t-t_{il})]^2. {} \end{aligned} $$

(10)

Let $\hat G_{ilk} = (Z_{il}-\hat {\mu }(t_{il}))(Z_{ik}-\hat {\mu }(t_{ik}))$ be the raw covariance points. The local linear smoother for the covariance function c _X is defined as $\hat {c}_X = \hat {\tilde {\beta }}_{0}$, where

$$\displaystyle \begin{aligned} (\hat{\tilde{\beta}}_{0},\hat{\tilde{\beta}}_{1},\hat{\tilde{\beta}}_{2}) &= \underset{\tilde{\beta}_{0},\tilde{\beta}_{1},\tilde{\beta}_{2}}{\text{arg min}}\,\sum_{i=1}^{n}\sum_{1\leq l,k\leq N_i} K\left(\frac{t_{il}-t}{h_c}\right)K\left(\frac{t_{ik}-s}{h_c}\right) \\ &\quad \times [\hat{G}_{ilk}-\tilde{\beta}_{0}-\tilde{\beta}_{1}(t_{il}-t)-\tilde{\beta}_{2}(t_{ik}-s)]^2. {} \end{aligned} $$

(11)

Similar to the technique introduced in Yao et al. (2005a), the points $\hat G_{ill}, l=1\cdots ,N_i$ are not included in (11). Let $\mathcal {T}_1=[\text{inf}\{L_i \in \mathcal {T}, i=1,\cdots ,n\}+|\mathcal {T}|/4, \text{sup}\{R_i\in \mathcal {T}, i=1,\cdots ,n\}-|\mathcal {T}|/4]$ with $|\mathcal {T}|$ being the length of $\mathcal {T}$. The estimator of $\sigma ^2_X$ is defined as $\hat {\sigma }^2_X$ if $\hat {\sigma }^2_X>0$, otherwise $\hat {\sigma }^2_X=0$ with

$$\displaystyle \begin{aligned}\hat{\sigma}^2_X=2\int_{\mathcal{T}_1}(\hat{V}_X(t)-\tilde{G}(t))\text{dt}/|\mathcal{T}|,\end{aligned}$$

where $\hat {V}_X(t)$ is the local linear estimator using the points $\{\hat G_{ill}\}$, $\tilde {G}(t)$ is the estimate $\hat {c}_X(s,t)$ restricted to s = t (Staniswalis and Lee 1998; Yao et al. 2005a). The estimators of {λ _j, ϕ _j}_j≥1 are the corresponding solutions of the eigen-equations

$$\displaystyle \begin{aligned} \int_{\mathcal{T}} \hat{c}_X(s,t)\hat{\phi}_j(s) \text{ds} = \hat{\lambda}_j\hat{\phi}_j(t). \end{aligned} $$

Based on the K-L expansion of X _i, model (9) can be rewritten as

$$\displaystyle \begin{aligned} Z_{il} = \mu(t_{il}) + \sum_{j=1}^{\infty}U_{ij}\phi_{j}(t_{il}) + \varepsilon_{il}, \quad t_{il}\in O_i, i=1\cdots,n, l=1\cdots,N_i. \end{aligned} $$

Let ${\mathbf {X}}_i=(X_i(t_{i1}),\cdots ,X_i(t_{iN_i}))^T$, ${\mathbf {Z}}_i=(Z_{i1},\cdots ,Z_{iN_i})^T$, μ _i = (μ(t _i1), ⋯ , $\mu (t_{iN_i}))^T$, ${\mathbf {\phi }}_{ij}=(\phi _j(t_{i1}),\cdots ,\phi _j(t_{iN_i}))^T$. Assume that U _ij and ε _il are jointly Gaussian. Following Yao et al. (2005a), the best prediction of U _ij of the ith subject given the observations (Z _il, t _il), l = 1, ⋯ , N _i is obtained as

$$\displaystyle \begin{aligned} \tilde{U}_{ij} = \lambda_j\mathbf\phi^T_{ij}\mathbf\varSigma^{-1}_{{\mathbf{Z}}_i}({\mathbf{Z}}_i-\mathbf{\mu}_i), \end{aligned} $$

where $\mathbf \varSigma _{{\mathbf {Z}}_i}=\text{cov}({\mathbf {Z}}_i, {\mathbf {Z}}_i) = \text{cov}({\mathbf {X}}_i, {\mathbf {X}}_i) + \sigma ^2_X {\mathbf {I}}_{N_i}$ with identity matrix ${\mathbf {I}}_{N_i}$. That is, the (u, v)th element of $\mathbf \varSigma _{{\mathbf {Z}}_i}$ is $(\mathbf \varSigma _{{\mathbf {Z}}_i})_{u,v} = c_X(t_{iu}, t_{iv}) + \sigma ^2_XI_{uv}$ with I _uv = 1 if u = v, and 0 otherwise. Then the estimator of U _ij is given through substituting μ, λ _j, ϕ _j with $\hat {\mu }, \hat {\lambda }_j, \hat {\phi }_j$ as

$$\displaystyle \begin{aligned} \hat{U}^{\mathrm{WME}}_{ij} = \hat{\lambda}_j\hat{\mathbf\phi}^T_{ij}\hat{\mathbf\varSigma}^{-1}_{{\mathbf{Z}}_i}({\mathbf{Z}}_i-\hat{\mathbf{\mu}}_i), {} \end{aligned} $$

(12)

where the (u, v)th entry of $\hat {\mathbf \varSigma }_{{\mathbf {Z}}_i}$ is $(\hat {\mathbf \varSigma }_{{\mathbf {Z}}_i})_{u,v} = \hat {c}_X(t_{iu}, t_{iv}) + \hat {\sigma }^2_X I_{uv}$. Replacing $\hat {U}_{ij}$ in (2) with $\hat {U}^{\text{WME}}_{ij}$, we then get the estimator $\hat {\gamma }^{\text{WME}}$ of γ from (3)

$$\displaystyle \begin{aligned} \hat{\gamma}^{\mathrm{WME}}(t) = \sum_{j=1}^{m} \hat{\gamma}_j \hat{\phi}_j, \end{aligned} $$

where $\hat {\gamma }_j$ is the jth entry of $\hat {\mathbf \gamma }$ with $\hat {U}^{\text{WME}}_{ij}$ in (2).

Next, we give some theoretical results for $\hat {\gamma }^{\text{WME}}(t)$. We assume the following regularity conditions which are similar to the assumptions in Kneip and Liebl (2020), Yao et al. (2005b).

• (B1) The observational points {t _il, l = 1, ⋯ , N _i} given O _i for the ith subject are i.i.d. random variables with pdf $f_{t|O_i}(u)>0$ for all $u\in O_i\subseteq \mathcal {T}$ and zero else. For the marginal pdf f _t of observation times t _ij, f _t(u) > 0 for all $u\in \mathcal {T}$.

• (B2) Let N = min{N _i, i = 1, ⋯ , n}. N ≍ n ^r with 0 < r < ∞, where a _n ≍ b _n means that there exists a constant 0 < L < ∞ such that a _n∕b _n → L as n →∞.

• (B3) h _μ → 0, h _c → 0, nNh _μ →∞, nMh _c →∞ as n →∞ with M = N ² − N.

• (B4) K is a second order kernel with compact support [−1, 1].

• (B5) Let G _ilk = (Z _il − μ(t _il))(Z _ik − μ(t _ik)). Define f _Zt, f _tt, f _Gtt as the joint pdf of (Z _il, t _il) on $\mathbb {R}\times \mathcal {T}$, $(t_{il_1},t_{il_2})$ on $\mathcal {T}^2$, (G _ilk, t _il, t _ik) on $\mathbb {R}\times \mathcal {T}^2$, respectively. All of the second derivatives of f _Zt, f _tt, f _Gtt are uniformly continuous and bounded. Moreover, f _t is uniformly continuous and bounded on $\mathcal {T}$.

• (B6) Let Λ = diag{λ ₁, ⋯ , λ _m}, Ξ = (λ ₁ ϕ _i1, ⋯ , λ _m ϕ _im)^T, $\varUpsilon = \mathbf {\varLambda }-\varXi \mathbf {\varSigma }^{-1}_{{\mathbf {Z}}_i}\varXi ^T$ and ς _n ≡trace(Υ). Denote $r_{\mu }=h^2_{\mu } + 1/\sqrt {nNh_{\mu }} + 1/\sqrt {n}$, $r_{c} = h^2_{c} + 1/\sqrt {nMh^2_{c}} + 1/\sqrt {n}$. υ _n ≡ mr _μ → 0, $\tau _n \equiv r_c(\sum _{j=1}^{m}\delta ^{-1}_j)\rightarrow 0$.

Theorem 2

Under the regularity conditions (A3), (A6), (B1)–(B6), we have that

$$\displaystyle \begin{aligned} \|\hat{\gamma}^{\mathit{\mbox{WME}}}-\gamma\|{}^2 = O_p(\upsilon_n+\tau_n+\varsigma_n+\theta_n). \end{aligned} $$

Theorem 2 gives the rate of convergence of the estimator $\hat {\gamma }^{\text{WME}}$ in the L ² sense. The rate of convergence of $\hat {\gamma }^{\text{WME}}$ depends on the sample size and bandwidths which is common for estimating curves or surface by local linear smoothers for functional data analysis (see Li and Hsing 2010). Related results of Theorem 2 can also be found in Yao et al. (2005b). The terms υ _n, τ _n are related to rates of convergence of estimators for the mean and covariance function by using local linear smoothers. The term ς _n is introduced by approximating U _ij with $\tilde {U}_{ij}$.

4 Simulation Studies

In this section, we use the simulated datasets to evaluate the finite sample properties of our proposed methods in Sect. 3. These studies are based on n ∈{50, 100, 200} i.i.d. samples $\{X_i,Y_i\}_{i=1}^n$ and equally spaced grid {t ₁, ⋯ , t ₃₀} on [0, 1] with t ₁ = 0, t ₃₀ = 1. For the ith functional observation X _i(t), the missing interval M _i takes the form [R _i − E _i, R _i + E _i], with $R_i=a_1T_{i1}^{1/2}$, E _i = a ₂ T _i2, where T _i1, T _i2 are independent random variables uniformly distributed on [0, 1], $a_1,a_2\in \mathbb {R}$. We consider (a ₁, a ₂) = (1.5, 0.2), (a ₁, a ₂) = (1.5, 0.4) with the expected missing length over the domain being 0.4 and 0.8, respectively. We set the intercept α = 0. To evaluate the performance of an estimator $\hat {\gamma }$ of γ, mean integrated square error (MISE) is used below as an evaluation criterion, given by,

$$\displaystyle \begin{aligned}\text{MISE} = \frac{1}{N}\sum_{l=1}^{N}\int_{0}^{1}(\hat{\gamma}_l(t)-\gamma(t))^2\text{dt},\end{aligned}$$

where N is the number of Monte Carlo replications.

For functional predictors {X _i} without measurement error, the trajectories are generated as follows. The simulated random function X _i has zero mean, the covariance function is generated from two eigenfunctions, $\phi _1(t)=\sqrt {2}\text{sin}(\pi t/2)$, $\phi _2(t)=\sqrt {2}\text{sin}(3\pi t/2)$. For the eigenvalues, we take λ ₁ = (π∕2)⁻², λ ₂ = (3π∕2)⁻², λ _k = 0, for k ≥ 3. The error 𝜖 _i in (1) is assumed to be standard normal. For the slope function γ in (1), we take the form γ(t) = ϕ ₁(t) + 3ϕ ₂(t). We compare the finite sample performance of our proposed method with the method that gives an estimator for γ through formula (2), (3) with deleting the incomplete functional observations in the datasets denoted as “SUB.” Moreover, the estimator of γ based on the original complete dataset is also considered in this scenario and denotes it as “ORI.” We conduct 1000 simulation runs in each setup. Table 1 reports the results.

Table 1 MISEs of the estimators of γ under different methods with 1000 Monte Carlo replications for functional predictors without measurement error

As shown in Table 1, in the scenario where incomplete functional predictors are observed without measurement error, the estimation method in Sect. 3.1 performs better than “SUB” method. This is because some useful information the dataset has will be lost if we delete them directly, while the “NME” method can take advantage of the whole information about the dataset. Specially, in each setting for (a ₁, a ₂), MISEs from the “NME” method have smaller values relative to the “SUB” method. These simulation results also demonstrate that MISEs decrease with increasing sample size n for these three methods. And MISEs increase with longer missing length on [0, 1] at fixed n indicating that a large error is introduced for the “NME” method in imputing missing scores of incomplete functional predictors through little available information from functional samples. In further, the difference of MISEs among these three methods are reduced with increasing sample size n, and the “NME” method still performs better than the “SUB” method, those imply the “NME” method is promising.

For functional predictors X _i with measurement error, they are generated according to Z _i(t _il) = X _i(t _il) + ε _il, l = 1, ⋯ , 30, as follows. We take $X_i(t)=\sum _{j=1}^{50}U_{ij}\phi _j(t)$ with U _ij = (−1)^j+1 j ^−1.1∕2 W _ij, where W _ij is uniformly distributed on $[-\sqrt {3}, \sqrt {3}]$, ϕ ₁(t) = 1, $\phi _j(t)=\sqrt {2}\text{cos}(j\pi t)$ for j ≥ 2. The additional random error ε _il, l = 1⋯ , 30 and the error 𝜖 _i in (1) are assumed to be normal with mean zero, variance 0.25. For the slope function γ, we take $\gamma =\sum _{j=1}^{50}\gamma _j\phi _j(t)$ with γ ₁ = 0.3, γ _j = 4(−1)^j+1 j ⁻² for j ≥ 2 (Hall and Horowitz 2007). We conduct 100 simulation runs in each setup. To demonstrate the superior performance of our proposed method in Sect. 3.2, we compare it with the other two methods after we get estimators of μ(t) and c _X(s, t) by solving the optimization problems (10), (11), respectively: one is that an estimator of γ is established by applying integration method to get the FPC scores $\hat {U}_{ij}$ in (2) instead of using formula (12), denoted as “IN”; the other one is that an estimator of γ is obtained by using the method in Sect. 3.1 with dataset {Z _i, Y _i} with measurement error being ignored. The results are summarized in Table 2.

Table 2 MISEs of the estimators of γ under different methods with 100 Monte Carlo replications for functional predictors with measurement error

We find from Table 2 that the “WME” method has the best performance relative to the other two methods in each setup, and the gains are dramatic when switching from the “NME” method to the “WME” method with the “NME” method ignoring observation errors for functional predictors. Specifically, for the case of n = 100, comparing with the “NME” method, the MISEs are reduced by 74%, 68% using the “WME” method with (a ₁, a ₂) = (1.5, 0.2) and (a ₁, a ₂) = (1.5, 0.4), respectively. For the “IN” method, it provides a reasonable estimator for γ and has better performance than the “NME” method, but nevertheless the “WME” method still performs better than “IN” method with improvement of 25%, 32% with respect to (a ₁, a ₂) = (1.5, 0.2) and (a ₁, a ₂) = (1.5, 0.4). In addition, these simulation results show that the MISEs decrease with increasing sample size n that is consistent with the derived theoretical results.

To sum up, in the scenario that incomplete functional predictors observed without measurement error, the “NME” method taking advantage of the whole information of the dataset produces a better estimator compared with the “SUB” method; in the scenario that incomplete functional predictors observed with measurement error, the “WME” method is preferred for giving the smallest MISE relative to the “IN” and “NME” methods. Both MISEs of the estimators of γ decrease with increasing sample size n that is consistent with the derived theoretical properties.

5 Real Data Analysis

A real diffusion tensor imaging (DTI) dataset considered here is from NIH Alzheimer’s Disease Neuroimaging Initiative (ADNI) study with 212 subjects, and is obtained through http://adni.loni.usc.edu/. The primary goal of ADNI study is to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), biological markers, and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). DTI obtained using mathematical method to represent the anisotropic diffusion of the water molecule in brain organization can be used to learn MCI and AD. The concrete measure of anisotropy includes fractional anisotropy (FA), relative anisotropy (RA), Volume ratio (VR), and FA is commonly adopted for its advantage in contrast ratio of grey-white matter. More details about preprocessing and methods of this study can be found in Zhu et al. (2012) and Yu et al. (2016).

Our main interest is characterizing the dynamic relationship between FA and mini-mental state examination (MMSE) score which is seen as a reliable and valid clinical measure in quantitatively assessing the severity of cognitive impairment. FA is measured at 83 equally spaced grid along the corpus callosum (CC) fiber tract that is the largest fiber tract in human brain, is responsible for much of the communication between two hemispheres, and connects homologous areas in two cerebral hemispheres.

To demonstrate the usefulness of the proposed method in Sect. 3.1, we artificially delete some observed points of FA, and then compare the estimator of γ obtained by using these incomplete functional observations with the estimator obtained by applying original complete dataset. For the ith FA curve, the missing domain has the same form with the interval given in Sect. 4 with (a ₁, a ₂) = (1.5, 0.2) and (a ₁, a ₂) = (1.5, 0.4). A part of complete and incomplete individual trajectories are displayed in Fig. 1.

Fig. 1

A part of complete (left) and incomplete (right) FA curves with mean function (purple line)

Estimators of functional coefficient obtained by both complete and incomplete FA dataset are illustrated in Fig. 2. It shows that estimators obtained by incomplete dataset with different missing domain (red line and green line) are similar to the estimator obtained from original complete dataset (blue line). This reveals that the proposed framework is useful in getting an estimator for the model with incomplete functional predictors.

Fig. 2

Estimators of γ with different expected missing length on [0, 1]. Blue line: the estimator using original complete dataset; Red line: the estimator with (a ₁, a ₂) = (1.5, 0.2); Green line: the estimator with (a ₁, a ₂) = (1.5, 0.4)

Next, we focus on the problem of recovering the missing parts $X_{iM_i}$ of X _i. Assume that the infinite-dimensional process X _i is well approximated by the projection onto the function space $L^2(\mathcal {T})$ via the first m eigenfunctions (Yao et al. 2005a). In practice, the prediction for the trajectory X _i(t) of the ith subject using the first m eigenfunctions given in Sect. 3.1 can be approached by

$$\displaystyle \begin{aligned} \hat{X}_i(t) = \hat{\mu}^{\text{NME}}(t) + \sum_{k=1}^{m}\hat{U}^{(\rho)}_{ij}\hat{\phi}^{\text{NME}}_j(t). \end{aligned} $$

We randomly select four FA curves with different missing parts. The predicted profiles for these four curves are presented in Fig. 3, showing that the predicted profiles are close to the real part. This demonstrates the “NME” method by recovering the missing parts of incomplete trajectories encourages a better estimator comparing with the “SUB” method with deleting them directly.

Fig. 3

Predicted profiles for four randomly chosen FA curves with different missing parts with (a ₁, a ₂) = (1.5, 0.2). Missing parts of trajectories from left to right and top to down: missing in left side, middle side, right side, both left and right side. Blue point: real data point; Red line: predicted profile

6 Discussion

In this paper, we address the problem for getting estimators of γ in (1) with partially observed trajectories without and with measurement error. Basic elements of our approach are estimators of FPC scores for each partially observed trajectory. Specially, in the scenario that incomplete functional predictors observed without measurement error, we achieve it by modeling FPC scores of the missing part as linear functionals of the observed part of that trajectory. In the scenario where incomplete functional data is observed with measurement error, we obtain estimators of FPC scores via conditional expectation. Rates of convergence of the proposed estimators $\hat {\gamma }^{\text{NME}}$, $\hat {\gamma }^{\text{WME}}$ under different scenarios are established. We also compare the proposed methods with the “SUB” or “IN” method. We conclude from simulation studies that both the “NME” and “WME” methods borrowing strength from entire samples to get estimators of γ in model (1) perform well in practice.

The methods proposed here can be extended to other models in terms of functional regression with partially observed trajectories, such as partial functional linear regression (see Shin 2009). The framework established in this paper is based on the assumption that missing parts of trajectories are missing completely at random. In a number of applications, it is common to encounter that the underlying missing mechanism for dataset depends on systematic strategies (Liebl and Rameseder 2019) that clearly violate MCAR assumption. Extension to this scenario is also of interest and significance in practice.

Appendix

Lemma 1 (Kraus (2015), Proposition 1)

• (a) Let E∥X∥² < ∞ and assumption (A1) be satisfied. Then $\mbox E(||\hat \mu ^{\mathit{\mbox{NME}}}-\mu ||{ }^2)=O(n^{-1})$ for n →∞.

• (b) Let E∥X∥⁴ < ∞ and observation pattern (A2) holds. Then $\mbox E(||\hat C^{\mathit{\mbox{NME}}}_X-C_X||{ }_S^2)=O(n^{-1})$ for n →∞ (here ||⋅||_S denotes the Hilbert–Schmidt norm).

Lemma 2 (Kneip and Liebl (2020), Theorem 4.1)

Under the assumptions (B1)–(B5), we have that

• (a) $\mathit{\mbox{sup}}_{t\in \mathcal {T}}|\hat {\mu }^{\mathit{\mbox{WME}}}(t) - \mu (t)| = O_p(r_{\mu })$ with $r_{\mu }=h^2_{\mu } + 1/\sqrt {nNh_{\mu }} + 1/\sqrt {n}$.

• (b) $\mathit{\mbox{sup}}_{(s,t)\in \mathcal {T}^2}|\hat {c}^{\mathit{\mbox{WME}}}(s,t)-c_X(s,t)| = O_p(r_{\mu }+r_{c})$ with $r_{c} = h^2_{c} + 1/\sqrt {nMh^2_{c}} + 1/\sqrt {n}$.

Proof of Theorem 1 The following results can be derived from the theory developed by Bhatia et al. (1983):

$$\displaystyle \begin{aligned} &\text{sup}_{j\geq 1} |\hat{\lambda}_j^{\mbox{NME}} - \lambda_j| \leq \|\hat{C}^{\mbox{NME}}_X - C_X\|, \,\,\,\,\,\,\,\, \\ &\text{sup}_{j\geq 1} \delta_j\|\hat{\phi_j}^{\mbox{NME}} - \phi_j\| \leq 8^{1/2} \|\hat{C}^{\mbox{NME}}_X - C_X\|. {} \end{aligned} $$

(13)

Therefore, we obtain from Lemma 1,

$$\displaystyle \begin{aligned} &\text{sup}_{j\geq 1} |\hat{\lambda}^{\mbox{NME}}_j - \lambda_j| = O_p(n^{-1/2}),\\ &\text{sup}_{j\geq 1}\delta_j\|\hat{\phi}^{\mbox{NME}}_j - \phi_j\| = O_p(n^{-1/2}). {} \end{aligned} $$

(14)

Note that,

$$\displaystyle \begin{aligned} &\int_{\mathcal{T}} (\hat{\gamma}^{\mbox{NME}}(s)-\gamma(s))^2\text{ds}\\ &\quad = \int_{\mathcal{T}} \left\{\sum_{j=1}^{m-1}\left[\frac{n^{-1}\sum_{i=1}^{n}[Y_i\hat{U}^{\mbox{NME}}_{ij}]}{\hat{\lambda}^{\mbox{NME}}_j}\hat{\phi}^{\mbox{NME}}_j(s)-\frac{\text{E}[YU_j]}{\lambda_j}\phi_j(s) \right] \right\}^2\text{ds}\\ &\qquad + \int_{\mathcal{T}} \left\{\sum_{j=m}^{\infty}\frac{\text{E}[YU_j]}{\lambda_j}\phi_j(s) \right\}^2\text{ds}\\ &\qquad +2\int_{\mathcal{T}}\left\{\sum_{j=1}^{m-1}\left[\frac{n^{-1}\sum_{i=1}^{n}[Y_i\hat{U}^{\mbox{NME}}_{ij}]}{\hat{\lambda}^{\mbox{NME}}_j}\hat{\phi}^{\mbox{NME}}_j(s)-\frac{\text{E}[YU_j]}{\lambda_j}\phi_j(s) \right] \right\} \left\{\sum_{j=m}^{\infty}\frac{\text{E}[YU_j]}{\lambda_j}\phi_j(s)\right\} \text{ds}\\ &\quad := A_1(n) + A_2(n) +A_3(n). {} \end{aligned} $$

(15)

For simplicity, we suppress the notation on “NME.” Assumption (A6) implies that A ₂(n) → 0 as m →∞. For A ₃(n), Cauchy–Schwarz inequality implies that $A^2_3(n)\leq A^2_1(n) \times A^2_2(n)\overset {p}{\rightarrow }0$. Combing the result (14), and the formula (15), we see that the result of the theorem follows if we can get the convergence rate of $\hat {U}_{ij}$ of the trajectories per subject with $\hat {U}_{ij} = \hat {U}_{ijO_i} + \hat U^{(\alpha )}_{ijM_i}$.

Denote the estimates of $U_{ijM_i}$, $C_{O_iO_i}$, $C_{O_iM_i}$, $\phi _{jM_i}$ as $\hat U_{ijM_i(-i)}$, $\hat C_{O_iO_i(-i)}$, $\hat C_{O_iM_i(-i)}$, $\hat \phi _{jM_i(-i)}$ with deleting the ith curves X _i(t). Let $\tilde {\xi }^{(\rho )}_{ijM_i} = (C^{(\rho )}_{O_iO_i})^{-1}$ $C_{O_iM_i}\phi _{jM_i}$ with $C^{(\rho )}_{O_iO_i} = C_{O_iO_i}+\rho \mathcal {F}_{O_i}$, $\tilde {U}_{ijM_i}^{(\rho )} = \langle \tilde {\xi }^{(\rho )}_{ijM_i}, X_{iO_i} \rangle$, and the notation $\tilde {\xi }_{ijM_i}$ $\tilde {U}_{ijM_i}$ are corresponded to the symbols $\tilde {\xi }^{(\rho )}_{ijM_i}$,$\tilde {U}_{ijM_i}^{(\rho )}$ with ρ = 0. Since

$$\displaystyle \begin{aligned} \text{E}\left\|\hat U_{ijM_i}^{(\rho)}-\tilde{U}_{ijM_i}\right\|{}^2 =&~~ \text{E}\left\|\hat U_{ijM_i}^{(\rho)}-\tilde{U}_{ijM_i}^{(\rho)}+\tilde{U}_{ijM_i}^{(\rho)}-\tilde{U}_{ijM_i}\right\|{}^2\\ =&~~ 2\text{E}\left\|\hat U_{ijM_i}^{(\rho)}-\tilde{U}_{ijM_i}^{(\rho)}\right\|{}^2+ 2\left\|\tilde{U}_{ijM_i}^{(\rho)}-\tilde{U}_{ijM_i}\right\|{}^2\\ \leq &~~ 4\text{E}\left\|\hat U_{ijM_i}^{(\rho)}-\hat{U}_{ijM_i(-i)}^{(\rho)}\right\|{}^2 + 4\text{E}\left\|\hat U_{ijM_i(-i)}^{(\rho)}-\tilde{U}_{ijM_i}^{(\rho)}\right\|{}^2 \\ & + 2\left\|\tilde{U}_{ijM_i}^{(\rho)}-\tilde{U}_{ijM_i}\right\|{}^2, {} \end{aligned} $$

(16)

we then analyze the terms $\text{E}\|\hat U_{ijM_i}^{(\rho )}-\hat {U}_{ijM_i(-i)}^{(\rho )}\|{ }^2$, $\text{E}\|\hat U_{ijM_i(-i)}^{(\rho )}-\tilde {U}_{ijM_i}^{(\rho )}\|{ }^2$, $\|\tilde {U}_{ijM_i}^{(\rho )}-\tilde {U}_{ijM_i}\|{ }^2$ in turn. Let $\hat {\xi }_{ijM_i(-i)}^{(\rho )} = (\hat C^{(\rho )}_{O_iO_i(-i)})^{-1}\hat C_{O_iM_i(-i)}\hat {\phi }_{jM_i(-i)}$. Then

$$\displaystyle \begin{aligned} &\text{E}\|\hat U_{ijM_i(-i)}^{(\rho)}-\tilde{U}_{ijM_i}^{(\rho)}\|{}^2 \\ &\quad =~~ \text{E}\langle\hat{\xi}_{ijM_i(-i)}^{(\rho)}-\tilde{\xi}_{ijM_i}^{(\rho)},X_{iO_i}\rangle^2\\ &\quad =~~ \text{E}\{\text{E}[\langle\hat{\xi}_{ijM_i(-i)}^{(\rho)}-\tilde{\xi}_{ijM_i}^{(\rho)},X_{iO_i}\rangle^2| \{X_{kO_i}, k\neq i\}]\}\\ &\quad =~~ \text{E}||C_{O_iO_i}^{1/2}((\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}\hat C_{O_iM_i(-i)}\hat{\phi}_{jM_i(-i)}-( C^{(\rho)}_{O_iO_i})^{-1}C_{O_iM_i}\phi_{jM_i})||{}^2\\ &\quad \leq ~~ 4\left\{\text{E} ||C_{O_iO_i}^{1/2}(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1} (\hat C_{O_iM_i(-i)}-C_{O_iM_i})(\hat{\phi}_{jM_i(-i)} -\phi_{jM_i})||{}^2 \right.\\ &\qquad + \text{E} ||C_{O_iO_i}^{1/2}(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1} C_{O_iM_i}(\hat{\phi}_{jM_i(-i)}-\phi_{jM_i})||{}^2\\ &\qquad + \text{E} ||C_{O_iO_i}^{1/2}(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}(\hat C_{O_iM_i(-i)}-C_{O_iM_i})\phi_{jM_i}||{}^2\\ &\qquad \left.+ \text{E} ||C_{O_iO_i}^{1/2}((\hat C^{(\rho)}_{O_iO_i(-i)})^{-1} - (C^{(\rho)}_{O_iO_i})^{-1}) C_{O_iM_i}\phi_{jM_i}||{}^2 \right\}\\ &\quad := B_1+B_2+B_3+B_4. {} \end{aligned} $$

(17)

Let $\mathcal {F}_m=\{\frac {\lambda _m}{2}<\hat {\lambda }_m<\frac {3}{2}\lambda _m\}$. Suppose the event $\mathcal {F}_m$ holds. Otherwise, we have $\text{P}(|\hat {\lambda }_m-\lambda _m|\geq \frac {\lambda _m}{2})\leq \text{P}(\|\hat {C}^{\mbox{NME}}_X - C_X\|\geq \frac {\lambda _m}{2})\rightarrow 0$ from assumption (A4). We have the following results for terms B ₁ to B ₄ with the equality

$$\displaystyle \begin{aligned} \left(\hat C^{(\rho)}_{O_iO_i(-i)}\right)^{-1}-\left(C^{(\rho)}_{O_iO_i}\right)^{-1} =(\hat C_{O_iO_i(-i)}-C_{O_iO_i})\left( C^{(\rho)}_{O_iO_i}\right)^{-1}\left(\hat C^{(\rho)}_{O_iO_i(-i)}\right)^{-1}. \end{aligned} $$

For the term B ₁,

$$\displaystyle \begin{aligned} B_1 &\leq \text{E} \left[\left\|C_{O_iO_i}^{1/2}\right\|{}_2^2 \cdot \left\|\left(\hat C^{(\rho)}_{O_iO_i(-i)}\right)^{-1}\right\|{}_\infty^2 \cdot \left\|\hat C_{O_iM_i(-i)}-C_{O_iM_i}\right\|{}_2^2 \cdot \left\|\hat{\phi}_{jM_i(-i)}-\phi_{jM_i}\right\|{}^2\right]\\ &=O\left(n^{-2}\delta^{-2}_j\right)\cdot O(\rho^{-2}). \end{aligned} $$

Denote ∥⋅∥_∞ as the operator norm. For the term B ₂, under the assumption (A7), $\text{E} \|C_{O_iO_i}^{1/2}\|{ }_\infty ^2<\infty$ and the result (14), it is clear that

$$\displaystyle \begin{aligned} B_2 &\leq \text{E} \left[\left\|C_{O_iO_i}^{1/2}\right\|{}_\infty^2 \cdot\left\|\left(\hat C^{(\rho)}_{O_iO_i(-i)}\right)^{-1}C_{O_iM_i}\right\|{}_2^2 \cdot \left\|\hat{\phi}_{jM_i(-i)}-\phi_{jM_i}\right\|{}^2\right]\\ &\leq \sum_j\sum_k \frac{r_{M_iO_ijk}^2 }{(\lambda_{O_i O_i k}+\rho)^2}\cdot O\left(n^{-1}\delta^{-2}_j\right) = O\left(n^{-1}\delta^{-2}_j\right). \end{aligned} $$

For the term B ₃,

$$\displaystyle \begin{aligned} B_3 &\leq \text{E} \left[\|C_{O_iO_i}^{1/2}\|{}_2^2 \cdot\|(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}\|{}_\infty^2 \cdot \|\hat C_{O_iM_i(-i)}-C_{O_iM_i}\|{}_2^2 \cdot \|\phi_{jM_i}\|{}^2\right]\\ &= O(n^{-1}\rho^{-2}). \end{aligned} $$

Note that $\frac {\rho \lambda _{O_iO_ik}}{(\lambda _{O_iO_ik}+\rho )^2}<1$. Under the assumption (A7), we have that

$$\displaystyle \begin{aligned} B_4 &\leq \text{E} \left[\|C_{O_iO_i}^{1/2} \cdot (C^{(\rho)}_{O_iO_i})^{-1}\cdot (\hat C^{(\rho)}_{O_iO_i(-i)})^{-1} \cdot C_{O_iM_i}\|{}_2^2 \cdot \|\hat C_{O_iO_i(-i)}-C_{O_iO_i}\|{}_2^2 \cdot \|\phi_{jM_i}\|{}^2\right]\\ &\leq \left\{\sum_j\sum_k\frac{\rho\lambda_{O_iO_ik}}{(\lambda_{O_iO_ik}+\rho)^2}\cdot \frac{{r_{O_iM_ijk}}^2}{(\lambda_{O_iO_ik}+\rho)^2}\cdot\rho^{-1}\right\}\cdot O(n^{-1})\\ &= O(n^{-1})\cdot O(\rho^{-1}). \end{aligned} $$

These results combined with (17) indicate

$$\displaystyle \begin{aligned} \text{E}\|\hat U_{ijM_i(-i)}^{(\rho)}-\tilde{U}_{ijM_i}^{(\rho)}\|{}^2 = O\left(n^{-1}\rho^{-2}+n^{-1}\delta^{-2}_j\right). {} \end{aligned}$$

(18)

We then analyze $\text{E}\|\hat U_{ijM_i}^{(\rho )}-\hat {U}_{ijM_i(-i)}^{(\rho )}\|{ }^2$,

$$\displaystyle \begin{aligned} \text{E}\|\hat U_{ijM_i}^{(\rho)}-\hat{U}_{ijM_i(-i)}^{(\rho)}\| =&~~ \text{E}\langle\hat{\xi}_{ijM_i}^{(\rho)}-\hat{\xi}_{ijM_i(-i)}^{(\rho)},X_{iO_i}\rangle\\ \leq &~~ \{\text{E}\|\hat{\xi}_{ijM_i}^{(\rho)}-\hat{\xi}_{ijM_i(-i)}^{(\rho)}\|{}^2\}^{1/2} \{\text{E}\| X_{iO_i} \|{}^2\}^{1/2}\\ \leq &~~L \{\text{E}\|\hat{\xi}_{ijM_i}^{(\rho)}-\hat{\xi}_{ijM_i(-i)}^{(\rho)}\|{}^2\}^{1/2}, {} \end{aligned} $$

(19)

where the last inequality holds from the finite second moment of X that is bounded by constant L. We also have,

$$\displaystyle \begin{aligned} \text{E} \|\hat{\xi}_{ijM_i}^{(\rho)}-\hat{\xi}_{ijM_i(-i)}^{(\rho)}\|{}^2 = &~~ \text{E} \| \left((\hat C^{(\rho)}_{O_iO_i})^{-1}\hat C_{O_iM_i} - (\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}\hat C_{O_iM_i(-i)}\right) \hat{\phi}_{jM_i(-i)} \|{}^2\\ =&~~ \text{E}\| \left[\left((\hat C^{(\rho)}_{O_iO_i})^{-1}-(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}\right)\hat C_{O_iM_i}\right. \\ & +\left. (\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}(\hat C_{O_iM_i}-\hat C_{O_iM_i(-i)})\right]\hat{\phi}_{jM_i(-i)} \|{}^2\\ \leq & ~~2 \left\{\text{E}\| \left((\hat C^{(\rho)}_{O_iO_i})^{-1}-(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}\right)\hat C_{O_iM_i}\|{}^2 \right. \\ &+ \left.\text{E} \|(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}(\hat C_{O_iM_i}-\hat C_{O_iM_i(-i)})\|{}^2 \right.\}. {} \end{aligned} $$

(20)

Note that

$$\displaystyle \begin{aligned} \text{E} \|\hat C_{O_iM_i}-\hat C_{O_iM_i(-i)}\|{}^2 = O(n^{-2}), \end{aligned}$$

$$\displaystyle \begin{aligned} \text{E}\|\left((\hat C^{(\rho)}_{O_iO_i})^{-1}-(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}\right)\hat C_{O_iM_i}\|{}^2 = O(n^{-2}), \end{aligned}$$

$$\displaystyle \begin{aligned} \text{E} \|(\hat C^{(\rho)}_{O_iO_i(-i)})^{-1}(\hat C_{O_iM_i}-\hat C_{O_iM_i(-i)})\|{}^2 = O(n^{-2}\rho^{-2}). \end{aligned}$$

Combining formulas (19) and (20), we deduce that

$$\displaystyle \begin{aligned} \text{E}\|\hat U_{ijM_i}^{(\rho)}-\hat{U}_{ijM_i(-i)}^{(\rho)}\|{}^2 = O(n^{-2}\rho^{-2}). {} \end{aligned}$$

(21)

On the other hand,

$$\displaystyle \begin{aligned} \text{E} \parallel \tilde{U}_{ijM_i}^{(\rho)}-\tilde{U}_{ijM_i}\parallel^2=O(\rho), {} \end{aligned}$$

(22)

$$\displaystyle \begin{aligned} \text{var}(\tilde{U}_{ijM_i}-U_{ijM_i}) &= \langle\phi_{jM_i},C_{M_iM_i}\phi_{jM_i}\rangle - \langle\phi_{jM_i}, C_{M_iO_i}C^{-1}_{O_iO_i}C_{O_iM_i}\phi_{jM_i}\rangle \\ &:=V_{ij}. {} \end{aligned} $$

(23)

Therefore, with nρ ³ → 0 and the formulas (16), (18), (21)–(23), we have that

$$\displaystyle \begin{aligned} \text{E} \|\hat U_{ijM_i}^{(\rho)}-U_{ijM_i}\|{}^2=O\left(n^{-1}\rho^{-2} + n^{-1}\delta^{-2}_j + V_{ij}\right). \end{aligned}$$

Then the results are proved with nρ ³ → 0. Proof of Theorem 2 Let $\tilde {\mathbf {U}}_{i} = (\tilde {U}_{i1},\cdots , \tilde {U}_{im})^T$, U _i = (U _i1, ⋯ , U _im)^T. The covariance matrix of $\tilde {\mathbf {U}}_{i}$ is $\text{var}(\mathbf {{U}}_{i})=\varXi \boldsymbol {\varSigma }^{-1}_{{\mathbf {Z}}_i}\varXi ^T$ with $\varXi = \text{cov}(\tilde {\mathbf {U}}_{i},{{\mathbf {Z}}_i})=(\lambda _1{{\boldsymbol \phi }}_{i1},\cdots ,\lambda _m{{\boldsymbol {\phi }}}_{im})^T$. Moreover, $\text{var}(\tilde {\mathbf {U}}_{i}-{\mathbf {U}}_{i}) = \boldsymbol {\varLambda }-\varXi \boldsymbol {\varSigma _{{\mathbf {Z}}_i}}\varXi ^T$. Combining these results with formulas (14), (12) and the results of Lemma 2, the result of Theorem 3 is obtained by replacing $\hat {U}^{\text{NME}}_{ij}$ with $\hat {U}^{\text{WME}}_{ij}$ in (15) with assumptions (B1)–(B6).