Abstract
In this paper, we propose to solve semidefinite nonlinear complementarity problems (NCP) associated to a nonlinear matrix function , by a quasi-Newton method. For this, we reformulate this problem as a smooth nonlinear matrix equation by using a new smooth NCP matrix function, then we apply a quasi-Newton method for solving this matrix equation. We prove the local superlinear convergence of our algorithm and we give some numerical examples to illustrate the efficiency of the proposed method.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Let \({\mathcal {S}}^{n}\) and \({\mathcal {S}}_{+}^{n}\) denote the space of \(n\times n\) symmetric matrices, and the cone of symmetric positive semidefinite matrices respectively. We endow \({\mathcal {S}}^{n}\) with the inner product and the Frobenius norm defined by
where \(X,S\in {\mathcal {S}}^{n}\) and Tr(·) stands for the trace of a matrix.
The notation \(A\succeq 0\) and \(A\succ 0\) means that A is symmetric positive semidefinite and symmetric positive definite, respectively.
Let \(F:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) be a continuously differentiable function. The Semidefinite Nonlinear Complementarity Problem associated to F (SDNCP(F)) is defined as follows:
Find \(X\in {\mathcal {S}}^{n}\) such that:
If F is an affine function of the form \(F(X)=L(X)+Q\) where \( L:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) is a linear operator and \(Q\in {\mathcal {S}}^{n}\), then SDNCP(F) is called a semidefinite linear complementarity problem (SDLCP(F), for short).
Note that the SDNCP(F) is a generalization of the nonnegative orthant nonlinear complementarity problem (NCP) defined by: Find \(x\in {\mathbb {R}} ^{n}\) such that:
where f : \( {\mathbb {R}} ^{n}\rightarrow {\mathbb {R}} ^{n}\) is continuously differentiable vector function.
This problem arises naturally in the optimality conditions of semidefinite programming problems (SDP) involving inequality constraints and has wide applications in engineering, economics, management sciences and other fields (see [10, 12], for example). Some methods have been proposed to solve the SDNCP(F) problems, for example, interior-point methods and merit function methods in [11]. Chen and Tseng [13] proposed a so-called exact non-interior continuation (or smoothing-type Newton) method to solve SDNCP(F).
Another approach to solve the problem (SDNCP(F)) is to reformulate it as a matrix equation \(H\left( X,S\right) =0\) where \( H:{\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\) is defined by
and \(\phi :{\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) is an NCP function, that is:
Most reformulations of the SDNCP(F) problem use nonsmooth NCP functions and the most popular NCP functions are Fischer–Burmeister function [6, 7, 9], defined by
and the natural residual function [2], defined by
where the matrix valued functions square root, min and absolute value are defined in the following section.
It is shown in [4], that the functions \(\phi _{\mathrm{FB}}\) and \(\phi _{\min }\) are not differentiable everywhere, but only strongly semismooth functions. To overcome the nonsmoothness, many authors [8, 13, 14] used smooth approximations of these functions. Smooth approximations of \(\phi _{\mathrm{FB}}\) and \(\phi _{\min }\) [2, 8, 13, 15], are given for a parameter \(\mu \in {\mathbb {R}} _{+}^{*}\), respectively by:
and
In this paper, we propose a new NCP function to give a smooth reformulation of the SDNCP(F) problem. This new NCP function is continuously differentiable and is defined by \(\phi _{\alpha }:{\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) where
The rest of the paper is organized as follows. In Sect. 2, we recall some useful preliminaries that will be needed in the sequel. We study some properties of the NCP function \(\phi _{\alpha }\) in Sect. 3. In Sect. 4, we develop our algorithm for solving the SDNCP(F) using the NCP function \(\phi _{\alpha }\) and a quasi-Newton method. For the sake of illustrating the effectiveness of our algorithm, some numerical experiments are reported in Sect. 5. We draw conclusions in Sect. 6.
2 Preliminaries
In this section, we recall the spectral definition of matrix valued function associated to a given real valued function and we present a classical result about its differentiability.
For any \(X\in {\mathcal {S}}^{n}\), let \(\lambda _{1}\left( X\right) \le \lambda _{2}\left( X\right) \le \cdots \le \lambda _{n}\left( X\right) \) be eigenvalues of X, then X admits a spectral decomposition of the form \( X=PD_{X}P^{\mathrm{T}}\) for some \(P\in {\mathcal {O}}^{n}\), where \({\mathcal {O}}^{n}\) denotes the set of \(P\in \mathcal { {\mathbb {R}} }^{n\times n}\) that are orthogonal and \(D_{X}=\hbox {diag}[\lambda _{1}\left( X\right) ,\ldots ,\lambda _{n}\left( X\right) ]\) denotes the \(n\times n\) diagonal matrix where \(\lambda _{i}\left( X\right) \) are diagonal elements.
Let \(f: {\mathbb {R}} \rightarrow {\mathbb {R}} \). We can define a corresponding matrix valued function \(F:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) by
where \(X=PD_{X}P^{\mathrm{T}}\) is the spectral decomposition of X.
Example 2.1
-
(1)
For \(f(t)=\sqrt{t},\) the corresponding function \(F:{\mathcal {S}}_{+}^{n}\rightarrow {\mathcal {S}}_{+}^{n}\) is defined by
$$\begin{aligned} F(X)=\sqrt{X}=P\hbox {diag}\left( \sqrt{ \lambda _{1}\left( X\right) },\ldots ,\sqrt{\lambda _{n}\left( X\right) }\right) P^{\mathrm{T}}. \end{aligned}$$is called square root function.
-
(2)
For \(f(t)=\min \left( 0,t\right) ,\) the corresponding function \( F:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) is defined by
$$\begin{aligned} F(X)=\min (0,X)=P\hbox {diag}\left( \min \left( 0,\lambda _{1}\left( X\right) \right) ,\ldots ,\min \left( 0,\lambda _{n}\left( X\right) \right) \right) P^{\mathrm{T}}. \end{aligned}$$is called the min function.
-
(3)
For \(f(t)=\left| t\right| ,\) the corresponding function \( F:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) is defined by
$$\begin{aligned} F(X)=\left| X\right| =P\text {diag}\left( \left| \lambda _{1}\left( X\right) \right| ,\ldots ,\left| \lambda _{n}\left( X\right) \right| \right) P^{\mathrm{T}}, \end{aligned}$$(2.2)
is called the absolute value function.
Remark 2.2
Note that the definition of absolute value function given by (2.2) is not applicable to the vectors, since here the absolute value is applied to the eigenvalues and not the entries of a matrix. Furthermore, we have
It is proved in [5, 7, 14] that the matrix valued function F inherited all topological properties of the function f, in particular we have:
Proposition 2.3
(See [7, 14] for the proof) Let \(f:{\mathbb {R}} \rightarrow {\mathbb {R}}\) and \(F:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) be the corresponding matrix valued function. Let \(X=PD_{X}P^{\mathrm{T}}\) be the spectral decomposition of X. Then
F is (continuously) differentiable at an \(X\in {\mathcal {S}}^{n}\) with eigenvalues \( \lambda _{1}\left( X\right) ,\ldots ,\lambda _{n}\left( X\right) \) if and only if f is (continuously) differentiable at \(\lambda _{1}\left( X\right) ,\ldots ,\lambda _{n}\left( X\right) .\) Moreover, \(F^{\prime }\left( X\right) \) is an operator given by
where \(\circ \) is the Hadamard product and \(f^{\left[ 1\right] }\left( D_{X}\right) \) is the matrix whose elements are:
3 Properties of the Function \(\phi _{\alpha }\)
In this section, we state some properties of the function \(\phi _{\alpha }\) given by (1.9). In particular, we show that this function is an NCP function, that is continuously differentiable and we calculate its derivative.
First, we remark that our new function \(\phi _{\alpha }\) can be expressed equivalently as follows:
or by the form:
with \(f(\cdot )=\min ^{2}(0,\cdot )\) and \(X+S=PDP^{\mathrm{T}}\) is the spectral decomposition of the symmetric matrix \(X+S\).
Proposition 3.1
The function \(\phi _{\alpha }\) defined in (1.9) is an NCP function for all \(0<\alpha \le 1,\) i.e.:
Proof
Let \(0<\alpha \le 1.\)
(1) Suppose that \(X\in {\mathcal {S}}_{+}^{n},\) \(S\in {\mathcal {S}}_{+}^{n}\), and \(\text {Tr}(XS)=0,\) then
and
Obviously, this implies that\(\ \phi _{\alpha }\left( X,S\right) =0\)
(2) Conversely, suppose that \(\phi _{\alpha }\left( X,S\right) =0\). Using the form (3.1) of the function \(\phi _{\alpha }\) and the fact that \(\phi _{\alpha }\left( X,S\right) =0,\) it follows that
So,
then \(XS+SX\in {\mathcal {S}}_{+}^{n}\).
Next, by expanding the right-hand side of (3.3) we obtain
since \(\left| X+S\right| ^{2}= \left( X+S\right) ^{2}\).
So, for all \(0<\alpha \le 1\), we get
and then
Let \(X+S=PDP^{\mathrm{T}}\) be the spectral decomposition of \(X+S,\) where \( D=\text {diag}\left( \lambda _{1},\ldots ,\lambda _{n}\right) ,\) then
then for all \(i=1,2,\ldots ,n,\) we have
therefore, \(X+S\in {\mathcal {S}}_{+}^{n}\) which implies that \(\min \left( 0,X+S\right) =0\), i.e: the eigenvalues of \(X+S\) are all nonnegative
Next, since \(\phi _{\alpha }\left( X,S\right) =0\) we get
So, we have shown that
Now, it remains to be shown that
The proof of (3.4) can be given by two methods:
Method 1:
Since \(XS+SX=0\) and \(X+S\in {\mathcal {S}}_{+}^{n}\), then:
so
hence
It is known that \(\phi _{\mathrm{FB}}\) is an NCP function [2], then
Method 2:
(1) Let \(X=LD_{X}L^{\mathrm{T}}\) be the spectral decomposition of X, where L is an orthogonal matrix and \(D_{X}=\text {diag}(\lambda _{1}\left( X\right) ,\ldots \lambda _{n}\left( X\right) ).\)
Since \(XS+SX=0\) and \(X+S\in {\mathcal {S}}_{+}^{n},\) we get
next, since \(L\,L^{\mathrm{T}}=L^{\mathrm{T}}\,L=I_n\), it follows from relations (3.5) that
Set \(B=(b_{ij})=L^{\mathrm{T}}SL\). Since \(D_{X}B+BD_{X}=0\), it follows that
in particular if we take \(i=j\), we obtain
Next, since \(D_{X}+B\in {\mathcal {S}}_{+}^{n}\) then it follows that
so, for all \(i=1,2,\ldots ,n\), we have
Obviously, this implies
then X is positive semidefinite matrix.
(2) In the same way, we prove that S is positive semidefinite matrix.
(3) We have \(\text {Tr}(XS)=\frac{1}{2}\text {Tr}(XS+SX)\), then \(\text {Tr}(XS)=0.\)
Therefore, we obtain
\(\square \)
Proposition 3.2
(Differentiability of the function \(\phi _{\alpha })\) The function \(\phi _{\alpha }\) defined by (1.9) is continuously differentiable everywhere and its derivative is given by :
for all \(U,V\in {\mathcal {S}}^{n},\) where \(X+S=PDP^{\mathrm{T}}\) is the spectral decomposition of \(X+S\) and :
Proof
From formula (1.9), we have
where \(\psi _{1}\left( X,S\right) =XS+SX\) and
with \(f(\cdot )=\min ^{2}(0,\cdot ).\)
It is clear that \(\psi _{1}\) is continuously differentiable everywhere. Now, by using the Proposition 2.3 and the fact that f is continuously differentiable everywhere, then the function \(\psi _{2}\) is continuously differentiable everywhere. Hence \(\phi _{\alpha }\) is continuously differentiable since it is the sum of two continuously differentiable functions. Now, we calculate the derivative of \(\phi _{\alpha }\)
(a) It is clear that
for all \(U,V\in {\mathcal {S}}^{n}.\)
(b) We compute the derivative of \(\psi _{2}\) in the following way
We put
where
and
then:
It is clear that the derivative of \(\Sigma \) at \(\left( X,S\right) \) is
for the derivative of \(\theta \left( Z\right) :\)
we have
where \(Z=PD_{Z}P^{\mathrm{T}}\) and \(D_{Z}=\text {diag}\left( \lambda _{1}\left( Z\right) ,\ldots ,\lambda _{n}\left( Z\right) \right) ,\) \(i=1,\ldots ,n\).
According to Proposition 2.3, the derivate of \(\theta \) is given by
Next, using the chain rule formula, we obtain
that is
Hence, the derivative of \(\psi _{2}\) is given by
where \(X+S=PDP\) is the spectral decomposition of \(X+S\) and
Finally, the derivate of the function \(\phi _{\alpha }\) is given by
for all \(U,V\in {\mathcal {S}}^{n}.\) \(\square \)
Remark 3.3
We can give the matrix function \(f^{\left[ 1\right] }\left( D\right) \) by the form
then we obtain
4 On the Applicability of Newton’s Method for Solving the SDNCP
Solving the SDNCP(F) problem comes back to solving the smooth equation \( H_{\alpha }\left( X,S\right) =0\), where
and \(\phi _{\alpha }\left( X,S\right) \) is defined by (1.9).
Since the functions \(\phi _{\alpha }\) and F are continuously differentiable then \(H_{\alpha }\) is a continuously differentiable function. Moreover, based on Proposition 3.2, the derivative (or Jacobian) operator \(H_{\alpha }^{\prime }\left( X,S\right) \) of \(H_{\alpha }\) at \(\left( X,S\right) \) is given by:
for all \(U,V\in {\mathcal {S}}^{n}\), where \(X+S=PDP^{\mathrm{T}}\) is the spectral decomposition of \(X+S.\)
To solve the equation \(H_{\alpha }(X,S)=0\) by Newton’s method, we must study the invertibility property of \(H_{\alpha }^{\prime }\left( X,S\right) \) at all \(\left( X,S\right) \) in \({\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\) near the solution. We need the following lemma:
Lemma 4.1
-
(i)
The operator of Lyapunov \(L_{A}:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n},\) defined by
$$\begin{aligned} \ L_{A}(X)=XA+AX, \end{aligned}$$is monotone (resp. strongly monotone) operator if \(A\succeq 0\) (resp. \(A\succ 0).\)
-
(ii)
If \(X,S\succ 0,\) then \(L_{X}\), \(L_{S}\) and \(L_{S}\circ L_{X}\) are strongly monotone and \(L_{X}\) and \(L_{S}\) are self-adjoint.
-
(iii)
Let \(B:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) be the operator defined by
$$\begin{aligned} \ B\left( U\right) =- \frac{1}{\alpha }P\left[ f^{\left[ 1\right] }\left( D\right) \circ \left( P\left( U\right) P^{\mathrm{T}}\right) \right] P^{\mathrm{T}}, \end{aligned}$$where \(X+S=PDP^{\mathrm{T}}\) is the spectral decomposition of \(X+S\). Then \(B=0\) if \(X+S\succeq 0,\) and B is strongly monotone operator if not.
Proof
For the proof of (i) and (ii) see Lemma 4.2 in [2].
(iii) According to Remark 3.3, it follows that
-
(1)
If \(X+S\succeq 0\), then
$$\begin{aligned} \left[ f^{\left[ 1\right] }\left( D\right) \right] _{ij}=0, \end{aligned}$$for all \(i,j=1,2,\ldots ,n.\) So,
$$\begin{aligned} B=0. \end{aligned}$$ -
(2)
If \(X+S\nsucceq 0\), then
$$\begin{aligned} \ \left[ f^{\left[ 1\right] }\left( D\right) \right] _{ij}<0, \end{aligned}$$for all \(i,j=1,2,\ldots ,n.\)
We have
so, for all \(U\in {\mathcal {S}}^{n}\) \(\left( U\ne 0\right) \), we have
Now, since
then
From the commutativity of the Hadamard product, we get
Then, the operator B is strongly monotone. \(\square \)
Proposition 4.2
(Invertibility of the derivative operator \(H_{\alpha }^{\prime }\left( X,S\right) \) for the monotone case) Suppose that F is a monotone function.
Then for all \(S\succ 0\) and \(X\succ 0,\) the derivative operator \(H_{\alpha }^{\prime }\left( X,S\right) \) defined by formula (4.2) is strongly monotone (so, invertible).
Proof
\(H_{\alpha }^{\prime }\left( X,S\right) \) is invertible operator \( \Leftrightarrow \! \left[ H_{\alpha }^{\prime }\left( X,S\right) \left( U,V\right) =\!0\Rightarrow \left( U,V\right) =\left( 0,0\right) \right] \)
we replace \(V=F^{^{\prime }}(X)U\) in the first equation:
To show that the operator \(H_{\alpha }^{\prime }\left( X,S\right) \) is strongly monotone (invertible), just show that the operator A is strongly monotone.
We have \(S\succ 0\) and \(X\succ 0\), then \(S+X\succ 0.\) So, by Lemma 4.1 we have
hence,
since \(L_{X}\) is strongly monotone, we put \(D=L_{X}^{-1}(U)\), then \(U=L_{X}(D)\)
Now, by using Lemma 4.1 (property (i) and (ii)) we have \(L_{X}\), \( L_{S} \) and \(L_{S}\circ L_{X}\) are strongly monotone and \(L_{X}\) and \(L_{S}\) are self-adjoint.
To show that A is strongly monotone we need to show that \(L_{X}\circ F^{\prime }(X)\circ L_{X})\) is at least monotone. Then for all \(D\in S^{n}\) (\(D\ne 0\)), we have
By the fact that \(F^{\prime }(X)\) is a monotone operator, we have
then, \(L_{X}\circ F^{\prime }(X)\circ L_{X}\) is monotone.
So, A is strongly monotone (sum of strongly monotone and monotone operator), so it is invertible. Consequently, the operator \(H_{\alpha }^{\prime }\left( X,S\right) \) is strongly monotone, so it is invertible. \(\square \)
Remark 4.3
If \(S\nsucc 0\) or \(X\nsucc 0,\) the derivative operator \(H_{\alpha }^{\prime }\left( X,S\right) \) may be not invertible. The following counter-example shows it.
Let \(F:{\mathcal {S}}^{2}\rightarrow {\mathcal {S}}^{2}\) be the function defined by
By using theorem 2.5 in [3], we have
It is clear that F is a strongly monotone function since \(F^{\prime }(X)\) is a strongly monotone operator that is
\(\left\langle F^{\prime }(X)U,U\right\rangle >0\), for all \(U\in {\mathcal {S}}^{2}.\)
Let \(X_{0}=\left( \begin{array}{cc} -1 &{} 0 \\ 0 &{} 2 \end{array} \right) \) and \(S_{0}=\left( \begin{array}{cc} 2 &{} 0 \\ 0 &{} -3 \end{array} \right) \) which are indefined matrices.
\(X_{0}+S_{0}=\left( \begin{array}{cc} 1 &{} 0 \\ 0 &{} -1 \end{array} \right) =PDP^{\mathrm{T}}\), where \(P=I_{2}\) and \(D=\left( \begin{array}{cc} 1 &{} 0 \\ 0 &{} -1 \end{array} \right) .\)
Let \(\left( U,V\right) =\left( \left( \begin{array}{cc} u_{11} &{} u_{12} \\ u_{12} &{} u_{22} \end{array} \right) ,\left( \begin{array}{cc} v_{11} &{} v_{12} \\ v_{12} &{} v_{22} \end{array} \right) \right) \), then
then for \(\alpha =1\),
then
Since \(\det \left( M\right) =0\), then the operator \(H^{\prime }\left( X_{0},S_{0}\right) \) is not invertible for \(X_{0}\) and \(S_{0}.\)
Remark 4.4
Since we can not guarantee the invertibility of \(H_{\alpha }^{\prime }\left( X_{k},S_{k}\right) \) at each iteration k, we can’t apply Newton’s method for solving the equation \(H_{\alpha }\left( X,S\right) =0.\) To avoid the inversion and the computation of the Jacobian of \(H_{\alpha }\) at each iteration, we propose to use a quasi-Newton method in Hilbertian space as defined in [1].
Recall that, quasi-Newton’s method for solving nonlinear equation
where \({\normalsize G:}\mathcal {X\rightarrow Y}\) is a Frechet-differentiable function and \({\mathcal {X}}\),\({\mathcal {Y}}\) are two Hilbert spaces, is defined by:
where the operators \(B_{k}\) are invertibles and approximate \(G^{\prime }\left( Z_{k}\right) \) at each iteration k.
In this paper, we use the Broyden’s method [1], to update \(B_{k}\) at each iteration, where the update formula is:
where \(W_{k}=Z_{k+1}-Z_{k},\) \(Y_{k}=G\left( Z_{k+1}\right) -G\left( Z_{k}\right) \) and \(\otimes \) is the dyadic product operator defined by
Remark 4.5
The notation \(\otimes \) is used in the literature for the Kronecker product, but here it is used for the dyadic (or tensor) operator defined by (4.3).
Note also that for all \(A, B \in {\mathcal {S}}^{n}\), the dyadic operator \( A\otimes C\) is a linear operator, but the Kronecker product of matrices A and B, (\(\text {Kron}(A,B)\) in Matlab notation) is a matrix of order \(n^{2}\times n^{2}\). The relation between the dyadic operator and the Kronecker product is given in [1, Prop. 13].
In the space of symmetric matrices \(({\mathcal {X}}={\mathcal {Y}}= {\mathcal {S}}^{n})\), the dyadic operator is given by
The following theorem shows that Broyden’s method converges superlinearly to the solution of the equation of \(G(Z)=0.\)
Theorem 4.6
(See [1] and references therein) Let \({\normalsize G:}{\mathcal {X}}\rightarrow {\mathcal {Y}}\) a continuously differentiable function in \(D\subset {\mathcal {X}},\) where D is a convex, open set. Assume that \(Z^{*}\in D\) is a zero point of G that \(G^{\prime }(Z^{*})\in L({\mathcal {X}},{\mathcal {Y}})\) is inversible and that \(G^{\prime }(\cdot )\) satisfies the Lipchitz condition :
for each \(Z\in D\).
Then, if \(Z_{0}\in D\) and \(B_{0}\in L({\mathcal {X}}\),\({\mathcal {Y}})\) are near \( Z^{*}\) and \(G^{\prime }(Z^{*})\) respectively, we have :
(1) the sequence \(\left\{ Z_{k}\right\} \) defined by the Broyden’s method is well-defined and converges superlinearly to the solution \(Z^{*}.\)
(2) Furthermore, \(B_{k}^{-1}\) exists for each k and the sequences \(\left\{ \left\| B_{k}\right\| \right\} \) and \(\left\{ \left\| B_{k}^{-1}\right\| \right\} \) are bounded.
In practical implementation, we use the Broyden’s method in the form
Now, we apply the Broyden’s method to solve the equation \(H_{\alpha }\left( X,S\right) =0\) defined by (4.1). This method is defined by
where \(B_{k}\) is updated by the following formula :
where \(W_{k}=\left( X_{k+1},S_{k+1}\right) -\left( X_{k},S_{k}\right) \) and \( Y_{k}=H_{\alpha }\left( X_{k+1},S_{k+1}\right) -H_{\alpha }\left( X_{k},S_{k}\right) . \)
Lemma 4.7
The operator \(B_{k}\) can be define by the formula
where \(\beta _{k} =\frac{{1}}{\left\langle {U}_{{k}} {,U}_{{k}}\right\rangle +\left\langle {V}_{{k}} {,V}_{{k}}\right\rangle }\).
Proof
We have
and \(B_{k}\left( W_{k}\right) =-H_{\alpha }\left( X_{k},S_{k}\right) \), then
where \(Y_{k}=H_{\alpha }\left( X_{k+1},S_{k+1}\right) -H_{\alpha }\left( X_{k},S_{k}\right) \) and \(W_{k}=\left( U_{k},V_{k}\right) .\)
Hence, we obtain
but \(H_{\alpha }\left( X_{k+1},S_{k+1}\right) =\left( \phi _{\alpha }\left( X_{k+1},S_{k+1}\right) ,F(X_{k+1})-S_{k+1}\right) \), then
Let \((U,V)\in {\mathcal {S}}^{n}\times {\mathcal {S}}^{n},\) by the definition of the dyadic product operator we have
then \(B_{k+1}\left( U,V\right) =B_{k}\left( U,V\right) +\frac{\left\langle {U}_{ {k}}{,U}\right\rangle +\left\langle {V}_{{k}} {,V}\right\rangle }{\left\langle {U}_{{k}}{,U}_{ {k}}\right\rangle +\left\langle {V}_{{k}}{,V}_{ {k}}\right\rangle }\big ( \phi _{\alpha }\left( X_{k+1},S_{k+1}\right) ,F(X_{k+1})-S_{k+1}\big ) ,\)
we put \(\beta =\frac{{1}}{\left\langle {U}_{{k}}{,U}_{{k}}\right\rangle +\left\langle {V}_{{k}}{,V }_{{k}}\right\rangle }\), then
\(B_{k+1}\left( U,V\right) =B_{k}\left( U,V\right) +\beta \left( \left\langle U_{k} {,}U\right\rangle +\left\langle V_{k}{,}V\right\rangle \right) \big ( \phi _{\alpha }\left( X_{k+1},S_{k+1}\right) ,F(X_{k+1})-S_{k+1}\big ) .\) \(\square \)
The following algorithm represents the quasi-Newton method applied to the equation \(H_{\alpha }\left( X,S\right) =0.\)
Algorithm 1: (Quasi-Newton’s method for SDNLCP(F)) | |
---|---|
Step 1: Initialization | |
Input a parameter \(\alpha \in (0,1]\) and a tolerance \( \varepsilon >0.\) Set \(k=0.\) | |
Initialize \(X_{0},S_{0}\in {\mathcal {S}}^{n}\) and \(B_{0}\in L\left( {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\right) \) be invertible. | |
Step 2: Stopping criteria | |
If \(\left\| H_{\alpha }\left( X_{k},S_{k}\right) \right\| _{F}<\varepsilon \), stop. Otherwise go to Step 3. | |
Step 3: Compute the matrices\(U_{k},V_{k}\ \in {\mathcal {S}}^{n}\) | |
Find the solution \(\left( U_{k},V_{k}\right) \in {\mathcal {S}}^{n}\) of the linear system: | |
\(\ B_{k}\left( U_{k},V_{k}\right) =-H_{\alpha }\left( X_{k},S_{k}\right) \) . | |
Step 4: Updating formula. | |
Compute \(B_{k+1}\in L\left( {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\right) \) by using formula (4.4). | |
Set \(\left( X_{k+1},S_{k+1}\right) =\left( X_{k},S_{k}\right) +\left( U^{k},V^{k}\right) .\) | |
Set \(k=k+1\) and go to Step 2. |
Remark 4.8
Since we must choose the operator \(B_{0}\) invertible (to guarantee that \( B_{k}\) is invertible for each iteration k), one possible choice is \( B_{0}=I \) (the identity operator in \(L\left( {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\right) )\) or \(B_{0}=H_{\alpha }^{\prime }\left( X_{0},S_{0}\right) \) for \(S_{0}\succ 0\) and \(X_{0}\succ 0\) (cf. Proposition 4.2).
Now, we will show the convergence of the Broyden’s method for the equation \(H_{\alpha }\left( X,S\right) =0\) defined by (4.1).
Theorem 4.9
(The convergence of Broyden’s method for the function \(H_{\alpha })\) Let \(H_{\alpha }:{\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\) be the function defined by (4.1). Suppose that Eq. (4.1) admits a strict solution denoted by \( \left( X^{*},S^{*}\right) ,\) i.e. (\(X^{*}\succ 0,\) \(S^{*}\succ 0\) and \(H_{\alpha }\left( X^{*},S^{*}\right) =0).\)
If \(\left( X_{0},S_{0}\right) \in {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\) and \(B_{0}\in L\left( {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\right) \) are chosen near \(\left( X^{*},S^{*}\right) \) and \(H_{\alpha }^{\prime }\left( X^{*},S^{*}\right) \) respectively such that \(B_{0}\) is invertible, then :
-
(1)
the sequence \(\left\{ \left( X_{k},S_{k}\right) \right\} \) defined by the Broyden’s method is well-defined and converges super-linearly to the solution \(\left( X^{*},S^{*}\right) .\)
-
(2)
the operator \(B_{k}^{-1}\) exists for all \(k\ge 0,\) and the sequences \(\left\{ \left\| B_{k}\right\| \right\} \) and \(\left\{ \left\| B_{k}^{-1}\right\| \right\} \) are bounded.
Proof
The proof is based on Theorem 4.6. We have
(1) \(H_{\alpha }^{\prime }\left( X^{*},S^{*}\right) \in L\left( {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\right) \) is an invertible operator, since \(H_{\alpha }^{\prime }\left( X^{*},S^{*}\right) \) is invertible for any \( X^{*}\succ 0\) and \(S^{*}\succ 0\) (Proposition 4.2).
(2) \(H_{\alpha }^{\prime }\left( X,S\right) \) is a Lipchitz operator for all \( \left( X,S\right) \in {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\) , since \(H_{\alpha }^{\prime }\left( X,S\right) \) is a Lipchitz operator everywhere. \(\square \)
4.1 Smoothing Newton’s Method for Solving SDNCP\(\left( F\right) \)
In this subsection, we develop the smoothing Newton’s method for solving SDNCP\(\left( F\right) \) problem.
Solving the SDNCP(F) problem by smoothing Newton’s method comes back to solving the smooth equation \(H_{\mu }\left( X,S\right) =0\) where \( H_{\mu }:{\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\times {\mathbb {R}} \rightarrow {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\) is defined by
Smoothing Newton’s method applied to the equation \(H_{\mu }\left( X,S\right) =0\) is defined by
where \(H_{\mu _{k}}^{\prime }\left( X_{k},S_{k}\right) \) is the derivative operator of \(H_{\mu _{k}}\) at \(\left( X_{k},S_{k}\right) .\)
In practical implementation, we use the smoothing Newton’s method in the form
So, at each iteration k of the smoothing Newton’s method we need to solve the system of linear equations defined in (4.7), where \(\left( \mu _{k}\right) \) is a decreasing sequence of positive numbers that tends to 0.
From [13, Lemma 2(c)], The derivative of \(\phi _{\mathrm{FB}}^{\mu }\) defined by (1.7) is given by
where \(C=\sqrt{X^{2}+S^{2}+2\mu ^{2}I}\) and \(L_{C}\) is a Lyapunov operator associated to C defined by
\(L_{C}\left( X\right) =CX+XC\)
Consequently the derivative of \(H_{\mu }\) is given by
In [13, Lemma 7], it is proved that if F is monotone then the operator \(H_{\mu }^{\prime }\left( X,S\right) \) is nonsingular for all \( \left( X,S\right) \in {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\) and \(\mu \in {\mathbb {R}} _{++}.\)
Now, for each iteration, the system of linear equations defined in (4.7 ) equivalent to
and
By applying \(L_{C_{k}}\) to (4.10), we obtain
then
and by (4.11), we have
substituting (4.13) in (4.12), we have
Algorithm 2: (Smoothing Newton’s method for SDNLCP(F)) | |
---|---|
Step 1: Initialization | |
Input a tolerance \(\varepsilon >0.\) Set \(k=0.\) | |
Initialize \(X_{0},S_{0}\in {\mathcal {S}}^{n}\) and \(\mu _{0}>0 \). | |
Step 2: Stopping criteria | |
If \(\left\| H_{\mu _{k}}\left( X_{k},S_{k}\right) \right\| _{F}<\varepsilon \), stop. Otherwise go to Step 3. | |
Step 3: Compute \(\left( U_{k},V_{k}\right) \) as follows: | |
Find the solution \(U_{k}\in S^{n}\) of linear system (4.14) | |
Compute the matrix \(V_{k}\in S^{n}\) by the relation (4.13) | |
Step 4: Updating formula. | |
Set \(\left( X_{k+1},S_{k+1}\right) =\left( X_{k},S_{k}\right) +\left( U^{k},V^{k}\right) .\) | |
Set \(k=k+1\) and go to Step 2. |
5 Numerical Experiments
In this section, some numerical experiments are given to show the performance of Algorithm 1 and the Smoothing Newton’s method (SNM for short) for solving SDNCP(F). We used a personal computer with 8.0 GB for random memory and Intel(R) core(TM) i7-4600M CPU 2.90 GHz to perform all numerical experiments. We used Windows 8 as operating system and Matlab R2017a to write the computer codes. For all examples, the stop criterion used in Algorithm 1 and SNM is \(er_1:=\Vert H_{\alpha }(X,S)\Vert _{F}<\varepsilon =10^{-10}\) and \(er_2:=\Vert H_{\mu }(X,S)\Vert _{F}<10^{-10}\) respectively or if the number of iterations is greater than 1000.
Example 5.1
Let \(F:S^{4}\rightarrow S^{4}\) be the function defined by
We can verify that the exact solution of the SDNCP(F) associated to this function is \((X^{*},S^{*})=(I_{4},0_{4})\). Furthermore, for all \( X=(x_{ij}),\,U=(u_{ij})\in S^{4}\), we have
Hence, for all \(U\in S^{4}\) such that \(U\ne 0_{4}\), we have
So, F is strongly monotone function since \(F^{\prime }(X)\) is strongly monotone operator.
We apply Algorithm 1 and SNM to solve the SDNCP(F), with \(\alpha =1\), \(B_{0}=H^{\prime }(X_{0},S_{0})\), \( \mu _k=(0.85)^k\), for different choices of initial guess \((X_{0},S_{0})\). The number of iterations k , CPU time in seconds and the values \(er1(k):=\Vert H_{\alpha }(X_{k},S_{k})\Vert \) and \(er2(k):=\Vert H_{\mu _k}(X_{k},S_{k})\Vert \) are listed in Tables 1 and 2.
The values of approximate solutions \((X_{k},S_{k})\) corresponding to different initial guesses \( (X_{0},S_{0})\) finded using Algorithm 1 are shown in Table 3 (Fig. 1).
where
Example 5.2
Let n be a positive integer and let \(a_{1},b_{1},a_{2},b_{2},\dots ,a_{n},b_{n}\) be a fixed real parameters such that
We define a function \(F:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) by
where for all \(X=(x_{ij})\in {\mathcal {S}}^{n}\),
By a simple verification, the exact solution of SDNCP(F) associated to function F is \((X^{*},S^{*})\), where
Using the definition of differentiability, we can prove that for all \( X=(x_{ij})\), \(U=(u_{ij})\in {\mathcal {S}}^{n}\), we have
Now, we take for example
We apply Algorithm 1 and SNM to solve the SDNCP(F), with \(\alpha =0.85\), \(\mu _k=\frac{1}{2^{k+1}}\), \((X_{0},S_{0})=(I_{n},I_{n})\) (for both Algorithm 1 and SNM) and \(B_{0}=H^{\prime }(X_{0},S_{0})\) for different values of dimension n. The number of iterations k , CPU time in seconds and the values \(er_1(k):=\Vert H_{\alpha }(X_{k},S_{k})\Vert \) and \(er_2(k):=\Vert H_{\mu }(X_{k},S_{k})\Vert \) are listed in Table 4. (Fig 2).
In the case \(n=8\), the approximate solution \((X_{32},S_{32})\) is
and
Note that, in this case \((n=8),\) the exact solution of SDNCP(F) is
Now, we repeat the same experiment with fixed dimension \(n=30\) and \((X_{0},S_{0})=(\lambda *\text {diag}(a),I_{n})\) where \(\lambda \in \{ 0.55 , 1 , 1.25 \}\), we get the results showing in Table 5 (Figs. 2, 3).
Example 5.3
Let \(Q\in S_{+}^{n}\) , \(n\ge 1\). We define a function \(F:{\mathcal {S}}^{n}\rightarrow {\mathcal {S}}^{n}\) by \(F(X)=X^{2}-Q\). It is clear that \((X^{*},S^{*})=(Q^{\frac{1 }{2}},0_{n})\) is the exact solution of SDNCP(F) associated to function F, where \(Q^{\frac{1}{2}}\) is the square root of the matrix Q.The function F is derivable on \({\mathcal {S}}^{n}\), and for all \((X,U)\in {\mathcal {S}}^{n}\times {\mathcal {S}}^{n}\), we have
According to Lemma 4.1, \(F^{\prime }(X)\) is strongly monotone. So F is also strongly monotone.
Now, we consider the following two choices of the matrix Q
where
We remark that when \(Q=Q_{1}\), the exact solution of the SDNCP(F) is
and when \(Q=Q_{2}\), then the exact solution of the SDNCP(F) is \((X^{*},S^{*})=(M,0_{10})\). We apply Algorithm 1 to solve the SDNCP(F), with \(n=10\), for the two choices \(Q=Q_{1}\) and \(Q=Q_{2}\) and different values of parameter \(\alpha \in \Omega :=\{0.10,0.25,0.50,0.75,1\}.\)
First case : \(Q=Q_{1}\). In the case, we take the initial guess \( (X_{0},S_{0})=(2*I_{10},0.1*I_{10})\) and \(B_{0}=H^{\prime }(X_{0},S_{0})\) . We get the results listed in Table 6.
From Table 6, we remark that for all \(\alpha \in \Omega \), we need 20 iterations to get desired approximate solution \((X_{20},S_{20})\), where
However, the optimal value of \(\alpha \) from \(\Omega \), which makes Algorithm more faster is 0.5.
Second case : \(Q=Q_{2}\). In the case, we take the initial guess \( (X_{0},S_{0})=(2*M,0.1*I_{10})\) and \(B_{0}=H^{\prime }(X_{0},S_{0})\).
From Table 7, we remark that Algorithm 1 has the same behavior for all \(\alpha \in \Omega \). We need 24 iterations to get desired approximate solution \((X_{24},S_{24})=(M+10^{-11}Z_{1}\,,\,10^{-11}Z_{2})\), where
and
Example 5.4
We show here an example in which Algorithm 1 converge while SNM diverge. Let n be a positive integer. Consider the problem posed in Example 4 such that \(Q=(P' P)^2\in {\mathcal {S}}^{n}_{+} \) where \(P=(p_{ij} )_{1\le i,j \le n}\) and
It is clear that the exact solution of the SDNCP(F) is \((X^{*},S^{*})=(P'P,0_{n})\). We apply Algorithm 1 and SNM to solve the SDNCP(F), with initial guess \( (X_{0},S_{0})=(1.25* P'P,0.1*I_{n})\) which is closed to \((X^{*},S^{*})\) and \(\alpha =0.5\) and \(\mu _k=\frac{1}{2^{k+1}}\). For different values of n we obtain results showing in Table 8.
For \(n=4\) we get using Algorithm 1 the following approximation \((X_{28} , S_{28})\) of exact solution \((P'P,0_{n})\) :
and
6 Conclusion
In this paper, we proposed a new smooth NCP matrix function and studied various properties of this function. Using these properties, we reformulated the SDNCP(F) problem as a smooth equation. We proved that Newton’s method can not be applied to solve this matrix equation since we can not guarantee that its Jacobian operator is invertible at each iteration. So, we applied a quasi-Newton’s method and proved that the convergence is superlinear. Also we give some developments of the smoothing Newton’s method for solving this problem. We concluded this paper by some numerical tests which confirm the theoretical results and demonstrate the efficiency of the proposed method, and we compared between the both methods.
References
Benahmed, B., Mokhtar-Kharroubi, H., de Malafosse, B., Yassine, A.: Quasi-Newton methods in infinite-dimensional spaces and application to matrix equations. J. Glob. Optim. (2011). https://doi.org/10.1007/s10898-010-9564-2
Kanzaw, C., Nagel, C.: Semidefinite programs: new search directions, smoothing-type methods and numerical results. SIAM J. Optim. 13(1), 1–23 (2002)
Sun, D., Sun, J.: Semismooth matrix-valued functions. Math. Oper. Res. 27(1), 150–169 (2002)
Sun, D., Sun, J.: Strong semismoothness of the Fischer–Burmeister SDC and SOC complementarity functions. Math. Program. 103, 575–581 (2005). https://doi.org/10.1007/s10107-005-0577-4
Buchholzer, H., Kanzow, C., Knabner, P., Kräutle, S.: The semismooth Newton method for the solution of reactive transport problems including mineral precipitation-dissolution reactions. Comput. Optim. Appl. 50, 193–221 (2011). https://doi.org/10.1007/s10589-010-9379-6
Han, L., Bi, S., Pan, S.: Nonsingularity of FB system and constraint nondegeneracy in semidefinite programming. Numer. Algorithms 62, 79–113 (2013). https://doi.org/10.1007/s11075-012-9567-9
Zhang, L., Zhang, N., Pang, L.: Differential properties of the symmetric matrix-valued Fischer–Burmeister function. J. Optim. Theory Appl. 153, 436–460 (2012). https://doi.org/10.1007/s10957-011-9962-8
Rui, S., Xu, C.: Inexact non-interior continuation method for monotone semidefinite complementarity problems. Optim. Lett. 6, 1411–1424 (2012). https://doi.org/10.1007/s11590-011-0337-8
Bi, S., Pan, S., Chen, J.-S.: Nonsingularity conditions for FB system of nonlinear SDPs. SIAM J. Optim. 21, 1392–1417 (2011)
Todd, M.J.: A study of search directions in primal-dual interior-point methods for semidefinite programming. Optim. Methods Softw. 11, 1–46 (1999)
Tseng, P.: Merit functions for semi-definite complementarity problems. Math. Program. 83, 159–185 (1998)
Wolkowicz, H., Saigal, R., Vandenberghe, L.: Handbook of Semidefinite Programming. Kluwer, Boston (2000)
Chen, X., Tseng, P.: Non-interior continuation methods for solving semidefinite complementarity problems. Math. Program. Ser. A 95, 431–474 (2003). https://doi.org/10.1007/s10107-002-0306-1
Chen, X., Qi, H., Tseng, P.: Analysis of nonsmooth symmetric-matrix-valued functions with applications to complementarity problems. SIAM J. Optim. 13(4), 960–985 (2003)
Huang, Z., Han, J.: Non-interior continuation method for solving the monotone semidefinite complementarity problem. Appl. Math. Optim. 47, 195–211 (2003). https://doi.org/10.1007/s00245-003-0765-7
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Behnam Hashemi.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ferhaoui, M., Benahmed, B. & Mouhadjer, L. A New Smooth NCP Function for Solving Semidefinite Nonlinear Complementarity Problems. Bull. Iran. Math. Soc. 48, 3909–3936 (2022). https://doi.org/10.1007/s41980-022-00725-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41980-022-00725-9
Keywords
- Semidefinite nonlinear complementarity problems
- NCP matrix function
- Matrix equation
- Quasi-Newton method
- Superlinear convergence