1 Introduction

We study the Schrödinger system with linear and nonlinear couplings(i.e., double couplings):

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u_1-\mu m_{11}(x)~u_1-\mu m_{12}(x)~u_2=-\beta u_1v_1^2-\beta u_1v_2^2, &{}\quad x\in \Omega ,\\ -\Delta u_2-\mu m_{21}(x)~u_1-\mu m_{22}(x)~u_2=-\beta u_2v_1^2-\beta u_2v_2^2, &{}\quad x\in \Omega ,\\ -\Delta v_1-\nu m_{11}(x)~v_1-\nu m_{12}(x)~v_2=-\beta v_1u_1^2-\beta v_1u_2^2, &{}\quad x\in \Omega ,\\ -\Delta v_2-\nu m_{21}(x)~v_1-\nu m_{22}(x)~v_2=-\beta v_2u_1^2-\beta v_2u_2^2, &{}\quad x\in \Omega ,\\ u_1,u_2,v_1,v_2\in H_0^1(\Omega ), \end{array} \right. \end{aligned}$$
(1.1)

where \(\Omega \subset {{{\mathbb {R}}}}^N(N=2{{ \text{ or } }}3)\) is a bounded radial domain. Moreover, \(\beta \) is a given positive nonlinear coupling constant, \(\mu ,\nu >0\) are undetermined linear coupling functions’ constants and the coefficient matrix \({{{{\varvec{M}}}}}=\left( m_{ij}(x)\right) _{2\times 2}\) satisfies the following conditions:

  1. (M1)

    \(m_{ij}\left( x\right) \in C\left( \overline{\Omega },{{{\mathbb {R}}}}\right) ,\forall i,j\in \{1,2\}\);

  2. (M2)

    \(m_{12}(x)=m_{21}(x)\), \(\forall x\in \overline{\Omega }\);

  3. (M3)

    \({{{{\varvec{M}}}}}=\left( m_{ij}\left( x\right) \right) _{2\times 2}\) is cooperative, i.e., \(m_{12}\left( x\right) >0,\forall x\in \Omega \);

  4. (M4)

    \(\max _{x\in \Omega }\max \{ m_{11}(x),m_{22}(x)\}>0\);

  5. (M5)

    \(m_{ij}\left( x\right) \left( i,j\in \{1,2\}\right) \) are radially symmetric, i.e., \(m_{ij}\left( x\right) =m_{ij}\left( |x|\right) ,x\in \Omega \).

System (1.1) arises from Bose–Einstein condensations with four hyperfine spin states and is also a natural model in nonlinear optics, see [1, 23, 35, 39] and the references therein. In the last 15 years, system (1.1) has attracted considerable attention. When linear coupling terms of system (1.1) don’t appear, there exist rich literatures to study the quantitative and qualitative properties of its solutions, here we only list some but not all literatures for the reader’s convenience(refer to the references and therein for more details). See [2, 3, 11, 12, 26, 28,29,30, 34, 41] for the existence of ground state or bound state solutions, see [6, 7, 10] for the bifurcation of the solutions, see [19, 21, 42, 47] for the singularly perturbed, see [15, 22, 27] for the semiclassical states, and see [8, 9, 36] for the normalized solutions. When system (1.1) admits the linear coupling only, we refer the reader to [4, 5, 18, 37] and the references therein. When system (1.1) admit both the linear coupling and nonlinear coupling terms(i.e., double couplings), thanks to our team’s sustainable studies, we have a relatively good understanding of the system. More specifically, the existence of bound state and ground state solutions have been investigated by the topological and variational methods in [14, 25, 31, 38], while the authors in [20, 44] study the bifurcation of synchronized solutions with parameters \(\kappa \) and \(\beta \) respectively. In [48], we obtain the symmetry results of ground state solutions and analyse its asymptotic behavior. Besides, we study the existence of normalized solutions to system (1.1) and their singularly perturbed limits in [32]. Recently, Ma et al. [33] investigate system (1.1) under the Neumann boundary conditions.

Based on the results obtained by [32], this paper aims to study the partial symmetry of the normalized solutions to system (1.1) and their symmetry breaking phenomena. With regard to the partial symmetry of solutions to elliptic systems, we have to mention the following works. In [45], Wang and Willem study a cooperative system and show that the least energy solutions are foliated Schwarz symmetry with respect to the same point. Besides, Tavares and Weth deal with a competitive system and prove that the ground state solutions are foliated Schwarz symmetric with respect to antipodal points in [43]. Recently, we investigate a doubly coupled system and obtain the partial symmetry of ground state solutions in [48]. Moreover, we prove that ground state solutions must be radial when the underlying domain is a ball with its center at the origin. However, it is worth noting that the literatures mentioned above deal with the cases that \(\mu \) and \(\nu \) are given and the coupling coefficients are the constants. In this paper, we assume that \(\mu \) and \(\nu \) are undetermined and the linear coupling coefficients are functions. For this case, we study the partial symmetry of the solutions to system (1.1).

Finally, we explain why we propose those assumptions on \({{{{\varvec{M}}}}}\). In 1999, Chang defined the principal eigenvalue of a class of elliptic system with weight \({{{{\varvec{M}}}}}\) and investigated its properties in [17]. Next in [32], we study the optimal partition for the principal eigenvalue of the elliptic system, which is approximated by the normalized solutions of system (1.1) as the nonlinear coupling \(\beta \rightarrow \infty \). Our paper is based on the results obtained in [32] and devoted to studying the partial symmetry of the normalized solutions to system (1.1) and their symmetry breaking phenomenons.

To state our main results, now we first give some notations. The vector means the column vector, that is, \({{{{\varvec{u}}}}}=(u_1,\ldots ,u_h)^\top \), \({{{{\varvec{u}}}}}^\top \) denotes the transpose of \({{{{\varvec{u}}}}}\). We define

$$\begin{aligned} ||u||_2^2=\int _\Omega |\nabla u|^2,~~ |u|_p^p=\int _\Omega |u|^p,\forall u\in H_0^1(\Omega ), \end{aligned}$$

and

$$\begin{aligned} \nabla {{{{\varvec{u}}}}}=(\nabla u_1,\ldots ,\nabla u_h)^\top , |\nabla {{{{\varvec{u}}}}}|^2=|\nabla u_1|^2+\cdots +|\nabla u_h|^2, \end{aligned}$$

and

$$\begin{aligned} |{{{{\varvec{u}}}}}|=(u_1^2+\cdots +u_h^2)^{1/2}. \end{aligned}$$

Here and hereafter, \({{{{\varvec{u}}}}}\ge 0\) (\({{{{\varvec{u}}}}}>0\)) means that \(u_i\ge 0\) (\(u_i>0\)), for every \(1\le i\le h\).

We define the energy functional by

$$\begin{aligned} J_\beta ({{{{\varvec{u}}}}},{{{{\varvec{v}}}}}):=\frac{1}{2}\int _\Omega |\nabla {{{{\varvec{u}}}}}|^2+\frac{1}{2}\int _\Omega |\nabla {{{{\varvec{v}}}}}|^2+\frac{\beta }{2}\int _\Omega |{{{{\varvec{u}}}}}|^2|{{{{\varvec{v}}}}}|^2 \end{aligned}$$

for every \({{{{\varvec{u}}}}}=(u_1,u_2),{{{{\varvec{v}}}}}=(v_1,v_2)\in \left[ H_0^1(\Omega ;{{{\mathbb {R}}}})\right] ^2\).

Since \(N=2,3\), by Sobolev embedding theorem, \(J_\beta ({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\) is well defined on \([H_0^1(\Omega ;{{{\mathbb {R}}}})]^2\times [H_0^1(\Omega ;{{{\mathbb {R}}}})]^2\), and \(J_\beta ({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\in C^1([H_0^1(\Omega ;{{{\mathbb {R}}}})]^2\times [H_0^1(\Omega ;{{{\mathbb {R}}}})]^2)\).

Let

$$\begin{aligned} \widetilde{\Sigma }:=\{{{{{\varvec{w}}}}}=(w_1,w_2)^\top \in [H_0^1(\Omega ;{{{\mathbb {R}}}})]^2:\int _\Omega {{{{\varvec{w}}}}}^\top {{{{\varvec{M}}}}}{{{{\varvec{w}}}}}=1\}, \end{aligned}$$

where \({{{{\varvec{w}}}}}^\top {{{{\varvec{M}}}}}{{{{\varvec{w}}}}}=m_{11}(x)w_1^2(x)+2m_{12}(x)w_1(x)w_2(x)+m_{22}(x)w_2^2(x)\). In order to obtain the least energy solutions of system (1.1), we study the energy minimization problem

$$\begin{aligned} c_\beta :=\inf _{({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\in \widetilde{\Sigma }\times \widetilde{\Sigma }}J_\beta ({{{{\varvec{u}}}}},{{{{\varvec{v}}}}}). \end{aligned}$$

If we set

$$\begin{aligned} \Sigma :=\{{{{{\varvec{w}}}}}=(w_1,w_2)^\top \in [H_0^1(\Omega ;{{{\mathbb {R}}}})]^2:\int _\Omega {{{{\varvec{w}}}}}^\top {{{{\varvec{M}}}}}{{{{\varvec{w}}}}}=1,w_i\ge 0,i=1,2\}, \end{aligned}$$

then we also have

$$\begin{aligned} c_\beta :=\inf _{({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\in \Sigma \times \Sigma }J_\beta ({{{{\varvec{u}}}}},{{{{\varvec{v}}}}}). \end{aligned}$$

To see this point, we refer the reader to reference [32] and omit the details of the proof.

Therefore, the assumption that \(w_i\)’s are nonnegative in the set \(\Sigma \) is a natural constraint and hence Lagrange multiplier rules can be applied. For positive least energy solutions, we have the following result, see [32, Theorem 1.1].

Theorem 1.1

Let \(\Omega \subseteq {{{\mathbb {R}}}}^N(N=2,3)\) be a smooth bounded domain. Suppose that the matrix \({{{{\varvec{M}}}}}\) satisfies \((M1)-(M4)\), then for every \(\beta >0\) there exists \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) achieving \(c_\beta \), which is a positive solution of the system (1.1) for some two Lagrange multipliers \(\mu _\beta>0,\nu _\beta >0\).

When the underlying domain \(\Omega \) is radial, a very natural problem is whether the positive solutions of (1.1) obtained by Theorem 1.1 inherit the symmetry or at least partial symmetry. To answer this question, we first recall the definition of foliated Schwarz symmetry. A positive function u defined on a radially symmetric domain \(\Omega \) is said to be foliated Schwarz symmetric with respect to \(p\in \partial B_1(0)\) if u depends only on \((r, \theta )=(|x|, \arccos (x\cdot p)/|x|)\) and is non-increasing in \(\theta \).

Now we state our main result on the partial symmetry of positive solutions of (1.1) obtained by Theorem 1.1.

Theorem 1.2

Let \(\Omega \subset {{{\mathbb {R}}}}^N(N=2{{ \text{ or } }}3)\) be a radially symmetric bounded domain with smooth boundary (i.e., \(\Omega =B_R(0)\) or \(B_R(0)\setminus \overline{B_r}(0),R>r>0\)), and suppose that the matrix \({{{{\varvec{M}}}}}\) satisfies \((M1)-(M5)\). For any given nonlinear coupling constant \(\beta >0\), we assume that \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) is a positive solution of (1.1) obtained by Theorem 1.1. Then there exists \(p\in \partial B_1(0)\) such that \(u_{1,\beta }\) and \(u_{2,\beta }\) are foliated Schwarz symmetric with respect to the same point p, while \(v_{1,\beta }\) and \(v_{2,\beta }\) are foliated Schwarz symmetric with respect to the antipodal point \(-p\).

Remark 1.3

From Theorem 1.2, we see that \(u_{1,\beta }\) and \(u_{2,\beta }\) tends to be synchronized, does also \(v_{1,\beta }\) and \(v_{2,\beta }\). But \({{{{\varvec{u}}}}}_\beta \) and \({{{{\varvec{v}}}}}_\beta \) as two groups behave mutually repulsed.

Compared to the known results (see [43, 45, 48]), the first difficulty lies in the fact that \(\mu \) and \(\nu \) are undetermined, which may make the operator of elliptic system indefinite. Our idea is to divide the vector solution into two groups, and to prescribe the masses of the two groups. The second difficulty results from the assumption that the linear coupling are functions, which needs to an extra effort in proving the foliated Schwarz symmetry. We overcome it by developing a kind of polarization inequality with weight, please see Lemma 2.4 and Corollary 2.5. In addition, the mixed effect of both linear coupling and nonlinear coupling gives rise to the obstacle in the proof. We deal with it by using the different polarization techniques for the two groups.

Although we obtain the partial symmetry of positive solutions of (1.1) founded by Theorem 1.1, there is an interesting problem: may \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) be radially symmetric functions, at least when \(\Omega \) is a ball? In the following, we give a negative answer for sufficiently large \(\beta \).

Theorem 1.4

Let \(\Omega \subset {{{\mathbb {R}}}}^N(N=2{{ \text{ or } }}3)\) be a radially symmetric bounded domain with smooth boundary (i.e., \(\Omega =B_R(0)\) or \(B_R(0)\setminus \overline{B_r}(0),R>r>0\)), and suppose that the matrix \({{{{\varvec{M}}}}}\) satisfies \((M1)-(M5)\). For every \(\beta >0\), we assume that \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) is a positive solution of (1.1) obtained by Theorem 1.1. Then for sufficiently large \(\beta >0\), \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) are not radially symmetric.

We briefly outline the ideas of the proof for Theorem 1.4. By Theorem 1.2, for any \(\beta >0\), we know \({{{{\varvec{u}}}}}_\beta \) and \({{{{\varvec{v}}}}}_\beta \) attain their maximum at a pair of antipodal points, say \((Q_\beta ,-Q_\beta )\). Since \(\Omega \) is bounded, up to a subsequence, we can assume \((Q_\beta ,-Q_\beta )\rightarrow (Q,-Q)\) as \(\beta \rightarrow +\infty \). Thanks to the asymptotic analysis of \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) as \(\beta \rightarrow +\infty \) given by our previous paper [32, Theorem 1.4], there exist limit profiles \({{{{\varvec{u}}}}}_\infty \) and \({{{{\varvec{v}}}}}_\infty \) such that \({{{{\varvec{u}}}}}_\beta (Q_\beta )\rightarrow {{{{\varvec{u}}}}}_\infty (Q),{{{{\varvec{v}}}}}_\beta (-Q_\beta )\rightarrow {{{{\varvec{v}}}}}_\infty (-Q)\). In virtue of the properties of limit profiles, we can show that \({{{{\varvec{u}}}}}_\infty (Q)>0,{{{{\varvec{v}}}}}_\infty (-Q)>0\) and \({{{{\varvec{u}}}}}_\infty (-Q)={{{{\varvec{v}}}}}_\infty (Q)=0\), which implies \({{{{\varvec{u}}}}}_\infty \) and \({{{{\varvec{v}}}}}_\infty \) are not radially symmetric functions, and hence \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) not radially symmetric solutions, at least for sufficiently large \(\beta \), which seems a new method.

This paper is organized as follows. In Sect. 2, we investigate partial symmetry of positive least energy solutions to system (1.1). In this section, we develop a kind of polarization inequality with weight (see Lemma 2.4 and Corollary 2.5) to give the proof of Theorem 1.2. Next, the symmetry breaking phenomenon has been analysed in Sect. 3. Compared with most of the literatures as before (for example, see [16, 24, 40]), our method is not based on comparing the energy of between the radial solutions and the non-radial solutions. In this article, we analyze the singularly perturbed limit profiles of these normalized solutions and prove that the normalized solutions are not radially symmetric at least for large nonlinear coupling constant \(\beta \).

2 Proof of Theorem 1.2

In this section, for the sake of clarity, we will drop the subscript \(\beta \) for \({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta ,\) \(\mu _\beta ,\nu _\beta \) and abbreviate them into \({{{{\varvec{u}}}}},{{{{\varvec{v}}}}},\mu ,\nu \). To show that \({{{{\varvec{u}}}}}\) and \({{{{\varvec{v}}}}}\) are foliated Schwarz symmetric, let us introduce some useful notations. As in [46], we define the sets

$$\begin{aligned} {{{\mathcal {H}}}}_0:=\left\{ H\subset {{{\mathbb {R}}}}^N: H {{ \text{ is } \text{ a } \text{ closed } \text{ half-space } \text{ in } }}{{{\mathbb {R}}}}^N{{ \text{ and } }}0\in \partial H\right\} \end{aligned}$$

and, for \(p\ne 0\), \({{{\mathcal {H}}}}_0(p)=\left\{ H\in {{{\mathcal {H}}}}_0:p\in {{\mathrm {int}}}(H)\right\} \). For each \(H\in {{{\mathcal {H}}}}_0\) we denote by \(\sigma _H:{{{\mathbb {R}}}}^N\rightarrow {{{\mathbb {R}}}}^N\) the reflection in \({{{\mathbb {R}}}}^N\) with respect to the hyperplane \(\partial H\), and define the polarization of a function \(u:\Omega \rightarrow {{{\mathbb {R}}}}\) with respect to H by

$$\begin{aligned} u_H(x):=\left\{ \begin{array}{cl} \max \{u(x), u(\sigma _H(x))\} \quad &{}x\in H\cap \Omega , \\ \min \{u(x), u(\sigma _H(x))\} \quad &{}x\in \Omega \backslash H. \end{array} \right. \end{aligned}$$

Moreover, we will call \(H\in {{{\mathcal {H}}}}_0\) dominant for u if \(u(x)\ge u(\sigma _H(x))\) for all \(x\in \Omega \cap H\)(or, equivalently, \(u_H(x)=u(x)\) for all \(x\in \Omega \cap H\)). On the other hand we will say that \(H\in {{{\mathcal {H}}}}_0\) is subordinate for u if \(u(x)\le u(\sigma _H(x))\) for all \(x\in \Omega \cap H\).

With the concepts above at hand, we first recall the following characterization of foliated Schwarz symmetry.

Lemma 2.1

[46, Proposition 2.4] Let \(u:\Omega \rightarrow {{{\mathbb {R}}}}\) be a continuous function. Then u is foliated Schwarz symmetric with respect to \(p\in \partial B_1(0)\) if and only if every \(H\in {{{\mathcal {H}}}}_0(p)\) is dominant for u.

Remark 2.2

By Lemma 2.1, noting that the definition of the polarization to a function, we also know that u is foliated Schwarz symmetric with respect to \(-p\in \partial B_1(0)\) if and only if every \(H\in {{{\mathcal {H}}}}_0(p)\) is subordinate for u.

Besides, we will need the following properties, see for instance [13, Lemma 2.2] and [46, Lemma 3.1].

Lemma 2.3

Let \(u:\Omega \rightarrow {{{\mathbb {R}}}}\) be a measurable function and \(H\in {{{\mathcal {H}}}}_0\).

  1. (i)

    If \(F:\Omega \times {{{\mathbb {R}}}}\rightarrow {{{\mathbb {R}}}}\) is a continuous function such that \(F(x,t)=F(y,t)\) for every \(x,y\in \Omega \) such that \(|x|=|y|\) and \(t\in {{{\mathbb {R}}}}\) and \(\int _\Omega |F(x,u(x))|dx<+\infty \), then \(\int _\Omega F(x,u_H)dx=\int _\Omega F(x,u)dx\).

  2. (ii)

    Moreover, if \(u\in H_0^1(\Omega )\) then also \(u_H\in H_0^1(\Omega )\) and \(\int _\Omega |\nabla u_H|^2=\int _\Omega |\nabla u|^2\).

For every \(H\in {{{\mathcal {H}}}}_0\) we denote by \(\widehat{H}\in {{{\mathcal {H}}}}_0\) the closure of the complementary half-space \({{{\mathbb {R}}}}^N\setminus H\). Then we give the following polarization inequalities with weight, which extends the previous results, see for instance [43, Lemma 4.5] and [45, Lemma 2.2].

Lemma 2.4

If \(P:\Omega \times {{{\mathbb {R}}}}\times {{{\mathbb {R}}}}\rightarrow {{{\mathbb {R}}}}\) is a continuous function such that \(P(x,t,s)=P(y,t,s)\) for every \(x,y\in \Omega \) such that \(|x|=|y|\) and \(t\in {{{\mathbb {R}}}}\). In addition, we suppose P is \(C^2\) with respect to ts and \(P_{ts}(x,t,s)=\frac{\partial ^2 P}{\partial t \partial s}(x,t,s)<0\) for every \(t,s>0\) and \(x\in \Omega \). Take \(u,v>0\) such that \(\int _\Omega P(x,u,v)dx<+\infty \). Then for every \(H\in {{{\mathcal {H}}}}_0\) we have that

$$\begin{aligned} \int _\Omega P(x,u_H,v_H)dx\le \int _\Omega P(x,u,v)dx\le \int _\Omega P(x,u_H,v_{\widehat{H}})dx. \end{aligned}$$

Proof

We claim that

$$\begin{aligned} P(x,a,c)+P(x,b,d)\le & {} P \left( x,\max \{a,b\},\min \{c,d\} \right) +P \left( x,\min \{a,b\},\max \{c,d\} \right) , \end{aligned}$$
(2.1)
$$\begin{aligned} P(x,a,c)+P(x,b,d)\ge & {} P \left( x,\max \{a,b\},\max \{c,d\} \right) +P \left( x,\min \{a,b\},\min \{c,d\} \right) , \end{aligned}$$
(2.2)

for every \(a,b,c,d>0\) and every \(x\in \Omega \). Thanks to the permutation of abc and d, we only suffice to consider two cases: \(a\ge b,c\ge d\) or \(a\ge b,c\le d\).

Case 1 \(a\ge b,c\ge d\). In this case, the inequality (2.2) trivially holds, and inequality (2.1) follows from

$$\begin{aligned} 0&\ge \int _{b}^{a}\int _{d}^{c}P_{ts}(x,t,s)dt ds=\int _{b}^{a}P_t(x,t,c)-P_t(x,t,d)dt\\&=P(x,a,c)-P(x,b,c)-P(x,a,d)+P(x,b,d). \end{aligned}$$

Case 2 \(a\ge b,c\le d\). In this case, the inequality (2.1) trivially holds, and inequality (2.2) follows from

$$\begin{aligned} 0&\ge \int _{a}^{b}\int _{c}^{d}P_{ts}(x,t,s)dt ds=\int _{a}^{b}P_t(x,t,d)-P_t(x,t,c)dt\\&=P(x,a,d)-P(x,b,d)-P(x,a,c)+P(x,b,c). \end{aligned}$$

By (2.1), we know

$$\begin{aligned} \int _\Omega P(x,u,v)dx&=\int _{\Omega \cap H} \left[ P(x,u(x),v(x))+P(\sigma _H(x),u(\sigma _H(x)),v(\sigma _H(x)))\right] dx\\&=\int _{\Omega \cap H} \left[ P(x,u(x),v(x))+P(x,u(\sigma _H(x)),v(\sigma _H(x)))\right] dx\\&\le \int _{\Omega \cap H} \left[ P(x,u_H(x),v_{\widehat{H}}(x))+P(x,u_H(\sigma _H(x)),v_{\widehat{H}}(\sigma _H(x)))\right] dx\\&=\int _\Omega P(x,u_H,v_{\widehat{H}})dx. \end{aligned}$$

Similarly, by (2.2), we get

$$\begin{aligned} \int _\Omega P(x,u,v)dx&=\int _{\Omega \cap H} \left[ P(x,u(x),v(x))+P(\sigma _H(x),u(\sigma _H(x)),v(\sigma _H(x)))\right] dx\\&=\int _{\Omega \cap H} \left[ P(x,u(x),v(x))+P(x,u(\sigma _H(x)),v(\sigma _H(x)))\right] dx\\&\ge \int _{\Omega \cap H} \left[ P(x,u_H(x),v_{H}(x))+P(x,u_H(\sigma _H(x)),v_{H}(\sigma _H(x)))\right] dx\\&=\int _\Omega P(x,u_H,v_H)dx. \end{aligned}$$

\(\square \)

Corollary 2.5

For \(u\in H_0^1(\Omega )\) and every radial function \(w\in C(\overline{\Omega },{{{\mathbb {R}}}})\) we have

$$\begin{aligned} \int _\Omega |\nabla u|^2 dx=\int _\Omega |\nabla u_H|^2 dx,\quad \int _\Omega w(x)u^q dx=\int _\Omega w(x)u_H^q dx, \end{aligned}$$

where \(q=2,4\). Moreover, if in addition \(w(x)>0,\forall x\in \Omega \), we have

$$\begin{aligned} \int _\Omega w(x)uvdx\le & {} \int _\Omega w(x)u_Hv_Hdx,\\ \quad \int _\Omega w(x)u_H^2v_{\widehat{H}}^2dx\le & {} \int _\Omega w(x)u^2v^2dx \le \int _\Omega w(x)u_H^2v_H^2dx \end{aligned}$$

for every \(u,v\in H_0^1(\Omega )\). In particular, when \(w(x)\equiv 1\), we obtain

$$\begin{aligned} \int _\Omega u^2 dx=\int _\Omega u_H^2 dx,\quad \int _\Omega u^4 dx=\int _\Omega u_H^4 dx \end{aligned}$$

and

$$\begin{aligned} \int _\Omega uvdx \le \int _\Omega u_Hv_Hdx,\quad \int _\Omega u_H^2v_{\widehat{H}}^2dx\le \int _\Omega u^2v^2dx \le \int _\Omega u_H^2v_H^2dx. \end{aligned}$$

Proof

The first result follows directly from Lemma 2.3. To show the second result, we first take \(P(x,t,s)=w(x)(t-s)^2\). Then it is easy to verify that P satisfies the assumptions of Lemma 2.4. So we obtain

$$\begin{aligned} \int _\Omega w(x)(u_H-v_H)^2dx\le \int _\Omega w(x)(u-v)^2dx, \end{aligned}$$

which together with the first result implies

$$\begin{aligned} \int _\Omega w(x)uvdx \le \int _\Omega w(x)u_Hv_Hdx. \end{aligned}$$

Similarly, if we take \(P(x,t,s)=w(x)(t^2-s^2)^2\), then we get

$$\begin{aligned} \int _\Omega w(x)(u_H^2-v_H^2)^2dx\le \int _\Omega w(x)(u^2-v^2)^2dx, \end{aligned}$$

and hence

$$\begin{aligned} \int _\Omega w(x)u^2v^2dx \le \int _\Omega w(x)u_H^2v_H^2dx. \end{aligned}$$

In addition, we also get

$$\begin{aligned} \int _\Omega w(x)(u_2-v_2)^2dx\le \int _\Omega w(x)(u_H^2-v_{\widehat{H}}^2)^2dx, \end{aligned}$$

which implies

$$\begin{aligned} \int _\Omega w(x)u_H^2v_{\widehat{H}}^2dx\le \int _\Omega w(x)u^2v^2dx. \end{aligned}$$

\(\square \)

In what follows, if \(({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\) is a positive minimizer for \(c_\beta \), we show that \(({{{{\varvec{u}}}}}_H,{{{{\varvec{v}}}}}_{\widehat{H}})\) is also a minimizer for \(c_\beta \) in virtue of the minimality of the energy that \(({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\) satisfies.

Proposition 2.6

If \(\left( {{{{\varvec{u}}}}}=(u_1,u_2)^\top ,{{{{\varvec{v}}}}}=(v_1,v_2)^\top \right) \) is a positive minimizer for \(c_\beta \), then also is \(\left( {{{{\varvec{u}}}}}_H=(u_{1,H},u_{2,H})^\top ,{{{{\varvec{v}}}}}_{\widehat{H}}=(v_{1,\widehat{H}},v_{2,\widehat{H}})^\top \right) \). Furthermore, we have

$$\begin{aligned} \int _\Omega m_{12}(x)u_{1,H}u_{2,H}dx& = {} \int _\Omega m_{12}(x)u_1u_2dx,\\ \quad \int _\Omega m_{12}(x)v_{1,\widehat{H}}v_{2,\widehat{H}}dx& = {} \int _\Omega m_{12}(x)v_1v_2dx \end{aligned}$$

and

$$\begin{aligned} \int _\Omega u_{1,H}^2v_{1,\widehat{H}}^2& = {} \int _\Omega u_{1}^2v_{1}^2,\quad \int _\Omega u_{1,H}^2v_{2,\widehat{H}}^2=\int _\Omega u_{1}^2v_{2}^2\\ \int _\Omega u_{2,H}^2v_{1,\widehat{H}}^2& = {} \int _\Omega u_{2}^2v_{1}^2,\quad \int _\Omega u_{2,H}^2v_{2,\widehat{H}}^2=\int _\Omega u_{2}^2v_{2}^2. \end{aligned}$$

Proof

Since \(u_i,v_i\in H_0^1(\Omega ),i=1,2\), we know by Lemma 2.3 (ii) that \(u_{i,H},v_{i,\widehat{H}},i=1,2\) are also in \(H_0^1(\Omega )\). Let us consider \((t{{{{\varvec{u}}}}}_H,s{{{{\varvec{v}}}}}_{\widehat{H}}) \in \Sigma \times \Sigma \) for some \(t>0,s>0\). According to Corollary 2.5 and the assumptions on \({{{{\varvec{M}}}}}\), we get

$$\begin{aligned} \int _\Omega m_{11}(x)u_{1,H}^2dx & = {} \int _\Omega m_{11}(x)u_{1}^2dx,\\ \quad \int _\Omega m_{22}(x)u_{2,H}^2dx & = {} \int _\Omega m_{22}(x)u_{2}^2dx,\\ \quad \int _\Omega m_{12}(x)u_{1,H}u_{2,H}dx\ge & {} \quad \int _\Omega m_{12}(x)u_1u_2dx. \end{aligned}$$

Since \({{{{\varvec{u}}}}}=(u_1,u_2)^\top \in \Sigma \), combining with the previous three equalities/inequality, we infer \(0<t\le 1\) to make \(t{{{{\varvec{u}}}}}_H\in \Sigma \). Similarly, we can get \(0<s\le 1\). Again by Corollary 2.5 and recalling the definition of \(c_\beta \), we have

$$\begin{aligned} c_\beta \le&J_\beta (t{{{{\varvec{u}}}}}_H,s{{{{\varvec{v}}}}}_{\widehat{H}})=\frac{t^2}{2}\int _\Omega |\nabla {{{{\varvec{u}}}}}_H|^2+\frac{s^2}{2}\int _\Omega |\nabla {{{{\varvec{v}}}}}_{\widehat{H}}|^2+\frac{\beta t^2s^2}{2}\int _\Omega |{{{{\varvec{u}}}}}_H|^2|{{{{\varvec{v}}}}}_{\widehat{H}}|^2\\ =&\frac{t^2}{2}\int _\Omega \left[ |\nabla u_{1,H}|^2+|\nabla u_{2,H}|^2\right] +\frac{s^2}{2}\int _\Omega \left[ |\nabla v_{1,\widehat{H}}|^2+|\nabla v_{2,\widehat{H}}|^2\right] \\&+\frac{\beta t^2s^2}{2}\int _\Omega \left[ u_{1,H}^2\left( v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2\right) +u_{2,H}^2\left( v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2\right) \right] \\ \le&\frac{t^2}{2}\int _\Omega \left[ |\nabla u_{1}|^2+|\nabla u_{2}|^2\right] +\frac{s^2}{2}\int _\Omega \left[ |\nabla v_{1}|^2+|\nabla v_{2}|^2\right] \\&+\frac{\beta t^2s^2 }{2}\int _\Omega \left[ u_{1}^2\left( v_{1}^2+v_{2}^2\right) +u_{2}^2\left( v_{1,}^2+v_{2}^2\right) \right] \\ \le&\frac{1}{2}\int _\Omega \left[ |\nabla u_{1}|^2+|\nabla u_{2}|^2\right] +\frac{1}{2}\int _\Omega \left[ |\nabla v_{1}|^2+|\nabla v_{2}|^2\right] \\&+\frac{\beta }{2}\int _\Omega \left[ u_{1}^2\left( v_{1}^2+v_{2}^2\right) +u_{2}^2\left( v_{1,}^2+v_{2}^2\right) \right] \\ =&c_\beta . \end{aligned}$$

That the equality above holds implies \(t=s=1\) and

$$\begin{aligned} \int _\Omega u_{1,H}^2v_{1,\widehat{H}}^2= &\, {} \int _\Omega u_{1}^2v_{1}^2,\quad \int _\Omega u_{1,H}^2v_{2,\widehat{H}}^2=\int _\Omega u_{1}^2v_{2}^2\\ \int _\Omega u_{2,H}^2v_{1,\widehat{H}}^2= &\, {} \int _\Omega u_{2}^2v_{1}^2,\quad \int _\Omega u_{2,H}^2v_{2,\widehat{H}}^2=\int _\Omega u_{2}^2v_{2}^2. \end{aligned}$$

Thus, we know \(J_\beta ({{{{\varvec{u}}}}}_H,{{{{\varvec{v}}}}}_{\widehat{H}})=c_\beta \). In addition, by using \(({{{{\varvec{u}}}}}_H,{{{{\varvec{v}}}}}_{\widehat{H}}) \in \Sigma \times \Sigma \), we also get

$$\begin{aligned} \int _\Omega m_{12}(x)u_{1,H}u_{2,H}dx= &\, {} \int _\Omega m_{12}(x)u_1u_2dx,\\ \quad \int _\Omega m_{12}(x)v_{1,\widehat{H}}v_{2,\widehat{H}}dx= &\, {} \int _\Omega m_{12}(x)v_1v_2dx. \end{aligned}$$

\(\square \)

Finally, with the previous preparations at hand, we now give the proof of our main result.

Proof of Theorem 1.2

For clarity, we divide the proof into three steps.

Step 1: Let \({{{{\varvec{u}}}}}_H,{{{{\varvec{v}}}}}_{\widehat{H}}\) be the polarization functions of \({{{{\varvec{u}}}}}\) and \({{{{\varvec{v}}}}}\). We show that \(({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\) and \(({{{{\varvec{u}}}}}_H,{{{{\varvec{v}}}}}_{\widehat{H}})\) satisfy the system (1.1) with the same Lagrange multipliers. As explained in the introduction, \(c_\beta \) can be written into

$$\begin{aligned} c_\beta =\inf _{({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\in \widetilde{\Sigma }\times \widetilde{\Sigma }}J_\beta ({{{{\varvec{u}}}}},{{{{\varvec{v}}}}}), \end{aligned}$$

where

$$\begin{aligned} \widetilde{\Sigma }=\{{{{{\varvec{w}}}}}=(w_1,w_2)^\top \in [H_0^1(\Omega ;{{{\mathbb {R}}}})]^2:\int _\Omega {{{{\varvec{w}}}}}^\top {{{{\varvec{M}}}}}{{{{\varvec{w}}}}}=1\}. \end{aligned}$$

Since \(({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\) is a positive minimizer for \(c_\beta \), by Proposition 2.6, we know \(({{{{\varvec{u}}}}}_H,{{{{\varvec{v}}}}}_{\widehat{H}})\) is also a minimizer for \(c_\beta \). Thus, there are by Lagrange multiplier rules \(\mu ,\nu ,\mu _H,\nu _H>0\) ( see [32] for their positivity) such that \(({{{{\varvec{u}}}}},{{{{\varvec{v}}}}})\) (\(({{{{\varvec{u}}}}}_H,{{{{\varvec{v}}}}}_{\widehat{H}})\) respectively) is a solution of system (1.1) with Lagrange multipliers \((\mu ,\nu )\) (\((\mu _H,\nu _H)\) respectively). Next we claim that

$$\begin{aligned} \mu =\mu _H,\quad \nu =\nu _H. \end{aligned}$$

In fact, when \(({{{{\varvec{u}}}}},{{{{\varvec{v}}}}},\mu ,\nu )\) solves (1.1), multiplying the first equation and the third equation of system (1.1) with \(u_1\) and \(v_1\) respectively, and then integrating over on \(\Omega \), we obtain

$$\begin{aligned} \mu \int _\Omega \left( m_{11}(x)u_1^2+m_{12}(x)u_1u_2\right) dx= &\, {} \int _\Omega |\nabla u_1|^2 dx+\beta \int _\Omega u_{1}^2\left( v_{1}^2+v_{2}^2\right) dx\\ \nu \int _\Omega \left( m_{11}(x)v_1^2+m_{12}(x)v_1v_2\right) dx= &\, {} \int _\Omega |\nabla v_1|^2 dx+\beta \int _\Omega v_{1}^2\left( u_{1}^2+u_{2}^2\right) dx. \end{aligned}$$

If \(({{{{\varvec{u}}}}}_H,{{{{\varvec{v}}}}}_{\widehat{H}},\mu _H,\nu _H)\) solves (1.1), taking a similar argument, we get

$$\begin{aligned} \mu _H\int _\Omega \left( m_{11}(x)u_{1,H}^2+m_{12}(x)u_{1,H}u_{2,H}\right) dx=&\int _\Omega |\nabla u_{1,H}|^2 dx\\&+\beta \int _\Omega u_{1,H}^2\left( v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2\right) dx\\ \nu _H\int _\Omega \left( m_{11}(x)v_{1,\widehat{H}}^2+m_{12}(x)v_{1,\widehat{H}}v_{2,\widehat{H}}\right) dx=&\int _\Omega |\nabla v_{1,\widehat{H}}|^2 dx\\&+\beta \int _\Omega v_{1,\widehat{H}}^2\left( u_{1,H}^2+u_{2,H}^2\right) dx. \end{aligned}$$

By Proposition 2.6, we easily see that \(\mu =\mu _H,\nu =\nu _H\).

Step 2: Take \(r>0\) such that \(\partial B_r(0)\subset \Omega \) and let \(p\in \partial B_1(0)\) be such that \(\max _{\partial B_r(0)} u=u(rp)\). Next we show that \({{{{\varvec{u}}}}}_H={{{{\varvec{u}}}}},{{{{\varvec{v}}}}}_{\widehat{H}}={{{{\varvec{v}}}}}\) for every \(H\in {{{\mathcal {H}}}}_0(p)\). Given \(H\in {{{\mathcal {H}}}}_0(p)\), by Step 1, we know

$$\begin{aligned} -\Delta u_1= &\, {} \mu m_{11}(x)u_1+\mu m_{12}(x)u_2-\beta u_1(v_1^2+v_2^2) \quad x\in \Omega ,\\ -\Delta u_{1,H}= &\, {} \mu m_{11}(x)u_{1,H}+\mu m_{12}(x)u_{2,H}-\beta u_{1,H}(v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2) \quad x\in \Omega . \end{aligned}$$

Therefore, we have

$$\begin{aligned} -\Delta (u_{1,H}-u_1)=&\mu m_{11}(u_{1,H}-u_1)+\mu m_{12}(u_{2,H}-u_2)\\&+\beta u_1(v_1^2+v_2^2)-\beta u_{1,H}(v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2)\\ =&\mu m_{11}(u_{1,H}-u_1)+\mu m_{12}(u_{2,H}-u_2)+\beta u_1(v_1^2+v_2^2)\\&-\beta u_1(v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2)\\&+\beta u_1(v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2)-\beta u_{1,H}(v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2). \end{aligned}$$

Let \(w(x):=u_{1,H}(x)-u_1(x),x\in \Omega \), then

$$\begin{aligned} -\Delta w+c(x) w=\mu m_{12}(x)(u_{2,H}-u_2)+\beta u_1\left[ (v_1^2-v_{1,\widehat{H}}^2)+(v_2^2-v_{2,\widehat{H}}^2)\right] , \end{aligned}$$
(2.3)

where \(c(x):=\beta (v_{1,\widehat{H}}^2+v_{2,\widehat{H}}^2)-\mu m_{11}(x)\). By Theorem 1.1, we know \(\mu >0\). By the assumption (M3) of \({{{{\varvec{M}}}}}\), it holds that \(m_{12}(x)>0,\forall x\in \Omega \). Besides, according to the definition of polarization functions and Theorem 1.1, we have

$$\begin{aligned} 0<u_i\le u_{i,H}\quad {{\text{ and }}}\quad 0< v_{i,\widehat{H}}\le v_i,i=1,2,\quad x\in \Omega \cap H. \end{aligned}$$

Therefore, we obtain

$$\begin{aligned} -\Delta w+c(x) w\ge 0, \quad w(x)\ge 0,\quad x\in \Omega \cap H. \end{aligned}$$

By the strong maximum principle, we get that either \(w>0\) or \(w\equiv 0\) in \(\Omega \cap H\). By the choice of p, we have that \(rp\in \Omega \cap H\) and that \(w(rp)=0\). And then it must be \(u_1=u_{1,H}\) and therefore \(w\equiv 0\) in \(\Omega \cap H\). Moreover, coming back to (2.3), we now see that

$$\begin{aligned} u_2\equiv u_{2,H},~~v_1\equiv v_{1,\widehat{H}},~~v_2\equiv v_{2,\widehat{H}},\quad x\in \Omega \cap H. \end{aligned}$$

Step 3: For every \(H\in {{{\mathcal {H}}}}_0(p)\), we get by Step 2 that

$$\begin{aligned} u_i\equiv u_{i,H},~~v_i\equiv v_{i,\widehat{H}},~~i=1,2,\quad x\in \Omega \cap H, \end{aligned}$$

which implies that H is dominant for \(u_1\) and \(u_2\), and is subordinate for \(v_1\) and \(v_2\). By Lemma 2.1 and Remark 2.2, we infer that \(u_{1}\) and \(u_{2}\) are foliated Schwarz symmetric with respect to p, while \(v_{1}\) and \(v_{2}\) are foliated Schwarz symmetric with respect to the antipodal point \(-p\). \(\square \)

3 Symmetry breaking

When considering the nonlinear coupling constant \(\beta \rightarrow +\infty \), we have the following results, see [32, Theorem 1.2 and 1.4].

Theorem 3.1

Let \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) be the positive least energy solution obtained by Theorem 1.1. Then there exists \(({{{{\varvec{u}}}}}_\infty ,{{{{\varvec{v}}}}}_\infty )\in \Sigma \times \Sigma \) such that, up to a subsequence, as \(\beta \rightarrow +\infty \),

  1. (i)

    \({{{{\varvec{u}}}}}_\beta \rightarrow {{{{\varvec{u}}}}}_\infty ,{{{{\varvec{v}}}}}_\beta \rightarrow {{{{\varvec{v}}}}}_\infty \) in \([H_0^1(\Omega )]^2\cap [C^{0,\alpha }(\overline{\Omega })]^2,\forall \alpha \in (0,1)\);

  2. (ii)

    \({{{{\varvec{u}}}}}_\infty \) and \({{{{\varvec{v}}}}}_\infty \) have disjoint supports, that is,

    $$\begin{aligned} u_{i,\infty }\cdot v_{j,\infty }\equiv 0,\forall i,j\in \{1,2\}. \end{aligned}$$
  3. (iii)

    \({{{{\varvec{u}}}}}_\infty \) and \({{{{\varvec{v}}}}}_\infty \) are Lipschitz continuous in \(\Omega \). And the sets \(\omega _{{{{{\varvec{u}}}}}_\infty }:=\{x\in \Omega :u_{1,\infty }^2(x)+u_{2,\infty }^2(x)>0\},\quad \omega _{{{{{\varvec{v}}}}}_\infty }:=\{x\in \Omega :v_{1,\infty }^2(x)+v_{2,\infty }^2(x)>0\}\) are open and connected;

  4. (iv)
    $$\begin{aligned} -\Delta {{{{\varvec{u}}}}}_\infty =\mu _\infty {{{{\varvec{M}}}}}{{{{\varvec{u}}}}}_\infty \quad {{\text{ in }}}~~ \omega _{{{{{\varvec{u}}}}}_\infty },\quad -\Delta {{{{\varvec{v}}}}}_\infty =\nu _\infty {{{{\varvec{M}}}}}{{{{\varvec{v}}}}}_\infty \quad {{\text{ in }}}~~ \omega _{{{{{\varvec{v}}}}}_\infty }, \end{aligned}$$

    and \({{{{\varvec{u}}}}}_\infty >0\) in \(\omega _{{{{{\varvec{u}}}}}_\infty }\), \({{{{\varvec{v}}}}}_\infty >0\) in \(\omega _{{{{{\varvec{v}}}}}_\infty }\), \(\mu _\infty =\lim _{\beta \rightarrow +\infty }\mu _\beta ,\nu _\infty =\lim _{\beta \rightarrow +\infty }\nu _\beta \), here \(\mu _\beta ,\nu _\beta \) are two Lagrange multipliers in Theorem1.1.

  5. (v)

    \(\overline{\Omega }=\overline{\omega _{{{{{\varvec{u}}}}}_\infty }}\cup \overline{\omega _{{{{{\varvec{u}}}}}_\infty }},\omega _{{{{{\varvec{u}}}}}_\infty }\cap \omega _{{{{{\varvec{v}}}}}_\infty }=\emptyset \). Moreover, \(\omega _{{{{{\varvec{u}}}}}_\infty }\ne \emptyset ,\omega _{{{{{\varvec{v}}}}}_\infty }\ne \emptyset \).

Remark 3.2

Although Theorem 3.1 (v) has not been pointed out explicitly in [32], we easily see this fact by checking the proof of Theorem 1.4 in [32].

Once we know the asymptotic behavior of \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) as \(\beta \rightarrow +\infty \), we can infer the shape of \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) from their limit profiles. Thus, we now can give the proof of Theorem 1.4.

Proof of Theorem 1.4

By Theorem 1.2, for any given \(\beta >0\), we know there exists \(p_\beta =p(\beta )\in \partial B_1(0)\) (here we emphasize the dependence on \(\beta \)) such that \(u_{1,\beta }\) and \(u_{2,\beta }\) are foliated Schwarz symmetric with respect to \(p_\beta \), while \(v_{1,\beta }\) and \(v_{2,\beta }\) are foliated Schwarz symmetric with respect to the antipodal point \(-p_\beta \). Therefore, recalling the definition of foliated Schwarz symmetry, we can assume that there exists \(\left\{ Q_\beta :\beta >0\right\} \subset \Omega \) such that

$$\begin{aligned} u_{i,\beta }(Q_\beta )=\max _{x\in \overline{\Omega }}u_{i,\beta }(x),\quad v_{i,\beta }(-Q_\beta )=\max _{x\in \overline{\Omega }}v_{i,\beta }(x),\quad i=1,2. \end{aligned}$$

Since \(\Omega \) is bounded, then there exists a subsequence \(\{\beta _k\}\) with \(\beta _k\rightarrow +\infty \) such that

$$\begin{aligned} Q_{\beta _k}\rightarrow Q\in \overline{\Omega },\quad {{\text{ as } }}k\rightarrow +\infty . \end{aligned}$$
(3.1)

On the other hand, by Theorem 3.1(i), we have

$$\begin{aligned} u_{i,\beta _k}\rightarrow u_{i,\infty }, \quad v_{i,\beta _k}\rightarrow v_{i,\infty }~~{{\text{ uniformly } \text{ for } }}x\in \overline{\Omega }, i=1,2 \quad {{\text{ as } }}k\rightarrow +\infty . \end{aligned}$$
(3.2)

Thus, we get

$$\begin{aligned} u_{i,\beta _k}(Q_{\beta _k})\rightarrow u_{i,\infty }(Q), \quad v_{i,\beta _k}(-Q_{\beta _k})\rightarrow v_{i,\infty }(-Q),~~i=1,2 \quad {{\text{ as } }}k\rightarrow +\infty . \end{aligned}$$

In fact, we take \(u_{1,\beta _k}(Q_{\beta _k})\rightarrow u_{1,\infty }(Q)\) for example to show this claim. For every \(\varepsilon >0\), by (3.1) and the continuity of \(u_{1,\infty }\) (see Theorem 3.1(iii)), there exists \(K_1>0\) such that

$$\begin{aligned} |u_{1,\infty }(Q_{\beta _k})-u_{1,\infty }(Q)|<\frac{\varepsilon }{2} \end{aligned}$$

whenever \(k>K_1\). In addition, there exists \(K_2>0\) from (3.2) such that when \(k>K_2\) we have

$$\begin{aligned} \left\| u_{1,\beta _k}-u_{1,\infty }\right\| _{L^\infty (\overline{\Omega })}<\frac{\varepsilon }{2}. \end{aligned}$$

Taking \(K=\max \{K_1,K_2\}>0\), we obtain

$$\begin{aligned} \left| u_{1,\beta _k}(Q_{\beta _k})-u_{1,\infty }(Q)\right| \le \left| u_{1,\beta _k}(Q_{\beta _k})-u_{1,\infty }(Q_{\beta _k})\right| +|u_{1,\infty }(Q_{\beta _k})-u_{1,\infty }(Q)|<\varepsilon \end{aligned}$$

whenever \(k>K\).

In what follows, we claim that \(u_{1,\infty }(Q)>0\). Since \(u_{1,\beta _k}>0\) in \(\Omega \), it holds that \(u_{1,\beta _k}(Q_{\beta _k})>0\) and hence \(u_{1,\infty }(Q)\ge 0\). If we assume \(u_{1,\infty }(Q)=0\), then we get \(u_{1,\beta _k}(Q_{\beta _k})=\left\| u_{1,\beta _k}\right\| _{L^\infty (\overline{\Omega })}\rightarrow 0\). Therefore it must have \(u_{1,\infty }\equiv 0\). By Theorem 3.1(iii) and (iv), we infer that

$$\begin{aligned} \omega _{{{{{\varvec{u}}}}}_\infty }=\{x\in \Omega :u_{1,\infty }^2(x)+u_{2,\infty }^2(x)>0\}=\{x\in \Omega :u_{1,\infty }(x)>0\}, \end{aligned}$$

which together with \(u_{1,\infty }\equiv 0\) implies \(\omega _{{{{{\varvec{u}}}}}_\infty }=\emptyset \). This contradicts the result of Theorem 3.1(v). So we have \(u_{1,\infty }(Q)>0\) and hence \(Q\in \Omega \). Similarly, we can also prove that \(u_{2,\infty }(Q)>0,v_{1,\infty }(-Q)>0,\) and \(v_{2,\infty }(-Q)>0\).

Finally, by Theorem 3.1(ii), we know \(u_{i,\infty }(-Q)=v_{i,\infty }(Q)=0,i=1,2\), which shows that \({{{{\varvec{u}}}}}_\infty \) and \({{{{\varvec{v}}}}}_\infty \) are not radially symmetric functions. From the strong convergence, we conclude that \(({{{{\varvec{u}}}}}_\beta ,{{{{\varvec{v}}}}}_\beta )\) are not radially symmetric solutions, at least for sufficiently large \(\beta \). \(\square \)