Rashba Spin–Orbit-Coupled Atomic Fermi Gases in a Two-Dimensional Optical Lattice

Abstract

The collective-mode excitation energy of a population-imbalanced spin–orbit-coupled atomic Fermi gas loaded in a two-dimensional optical lattice at zero temperature is calculated within the Gaussian approximation, and from the Bethe–Salpeter equation in the generalized random-phase approximation assuming the existence of a Sarma superfluid state. It is found that the Gaussian approximation overestimates the speed of sound of the Goldstone mode. More interestingly, the Gaussian approximation fails to reproduce the roton-like structure of the collective-mode dispersion which appears after the linear part of the dispersion in the Bethe–Salpeter approach. We investigate the speed of sound of a balanced spin–orbit-coupled atomic Fermi gas near the boundary of the topological phase transition driven by an out-of-plane Zeeman field. It is shown that the minimum of the speed of sound is located at the topological phase transition boundary, and this fact can be used to confirm the existence of a topological phase transition.

1 Introduction

Topological superfluidity is an interesting state of matter, partly because it is associated with quasiparticle excitations which are Majorana fermions. The basic physics behind the emergence of the Majorana fermion excitations is the existence of s-wave superfluidity, nonvanishing spin–orbit coupling (SOC), and Zeeman splitting. In this context, calculating the dispersion of the collective modes when pseudospin of atoms can couple with not only the effective Zeeman field, but also with the orbital degrees of freedom of atoms is an important and interesting problem by itself.

It is widely accepted among the optical lattice community that the attractive Fermi–Hubbard model captures the s-wave superfluidity of cold fermion atoms in optical lattices. According to this model, two fermion atoms of opposite pseudospins on the same site have an attractive interaction energy U, while the probability to tunnel to a neighboring site is given by the hopping parameter J:

$$\begin{aligned} H_U=-\sum _{<i,j>,\sigma }J_{\sigma }\psi ^\dag _{i,\sigma }\psi _{j,\sigma } -U\sum _i \widehat{n}_{i,\uparrow } \widehat{n}_{i,\downarrow }-\sum _{i,\sigma }\mu _\sigma \widehat{n}_{i,\sigma }. \end{aligned}$$

(1)

Here, $J_{\sigma }$ is the tunneling strength of the atoms between nearest-neighbor sites, and $\widehat{n}_{i,\sigma }=\psi ^\dag _{i,\sigma }\psi _{i,\sigma }$ is the density operator on site i. The Fermi operator $\psi ^\dag _{i,\sigma }$ ($\psi _{i,\sigma }$) creates (destroys) a fermion on the lattice site i with pseudospin projection $\sigma =\uparrow ,\downarrow$. The symbol $\sum _{<ij>}$ means sum over nearest-neighbor sites of the two-dimensional (2D) lattice.

In this paper we assume the existence of nonvanishing Rashba SOC in the xy plane and a Zeeman field along the z direction, so the Hamiltonian of the system is $\widehat{H}=\widehat{H}_{U}+\widehat{H}_\mathrm{SOC}+\widehat{H}_{Z}$. In the case of a 2D optical lattice the SOC part of the Hamiltonian is given by [1, 2]

$$\begin{aligned} \widehat{H}_\mathrm{SOC}=-\imath \lambda \sum _{<i,j>} \left( \psi ^\dag _{i,\uparrow }, \psi ^\dag _{j,\downarrow }\right) \left( \overrightarrow{\sigma }\times \mathbf d _{i,j}\right) _z\left( \begin{array}{l} \psi _{i,\uparrow }\\ \psi _{j,\downarrow }\\ \end{array} \right) , \end{aligned}$$

(2)

where $\lambda$ is the Rashba SO coupling coefficient, $\overrightarrow{\sigma }=(\sigma _x, \sigma _y,\sigma _z)$, $\sigma _{x,y,z}$ are the Pauli matrices, and $\mathbf d _{i,j}$ is a unit vector along the line that connects site j to i.

The out-of-plane Zeeman field is described by the term $\widehat{H}_{Z}$:

$$\begin{aligned} \widehat{H}_{Z}=h_{z}\sum _{i} \left( \psi ^\dag _{i,\uparrow }, \psi ^\dag _{i,\downarrow }\right) \sigma _{z}\left( \begin{array}{l} \psi _{i,\uparrow }\\ \psi _{i,\downarrow }\\ \end{array} \right) . \end{aligned}$$

(3)

In what follows, we study the collective-mode dispersion of species of Fermi atoms with equal, or imbalance, population in two pseudospin states loaded in a 2D optical lattice in the presence of both Zeeman field and nonvanishing Rashba type of spin–orbit coupling by applying functional integral technique which requires the representation of the Hubbard interaction in terms of squares of one-body charge and spin operators. It is possible to resolve the Hubbard interaction into quadratic forms of spin and electron number operators in an infinite number of ways by applying the Hubbard–Stratonovich transformation. If no approximations were made in evaluating the functional integrals, it would not matter which of the ways is chosen. When approximations are taken, the final result depends on a particular form chosen.

One of the most common ways to apply the Hubbard–Stratonovich transformation is to introduce the energy gap as an order parameter field. This allows us to integrate out the fermion fields and to arrive at an effective action. The next steps are to consider the state which corresponds to the saddle point of the effective action, and to write the effective action as a series in powers of the fluctuations and their derivatives. In the Gaussian approximation, the terms up to second order in the fluctuations and their derivatives are explicitly calculated. To the best of our knowledge, the Gaussian approximation is the only approximation that has been used to obtain the speed of sound in the presence of the SOC and the Zeeman field effects [3–10]. In a diagrammatic language, the Gaussian approximation can be derived by summation of the infinite sequences of graphs in the ladder approximation, and by neglecting the exchange interaction, which is represented by bubble diagrams [11].

The main goal of the present study is to go beyond the Gaussian approximation by taking into account both, the direct and the exchange interactions. This will be achieved by deriving the corresponding Bethe–Salpeter (BS) equation for the collective modes, which takes into account both, the ladder and the bubble diagrams. In particular, we use the generalized random-phase approximation (GRPA) for the collective excitations, in which the single-particle excitations are obtained in the mean field approximation (or by solving the Bogoliubov-de Gennes equations in a self-consistent fashion); while the collective modes are obtained by solving the BS equation in which the single-particle Green’s functions are calculated in Hartree–Fock approximation, and the BS kernel is obtained by summing ladder and bubble diagrams.

2 Collective Modes of an Imbalanced Rashba Spin–Orbit-Coupled Sarma Superfluid in an Optical Lattice

In this Section, we consider pairing between atoms with equal and opposite momenta. If the system is imbalanced, the corresponding states are known as the Sarma superfluid states [12]. Since the Sarma states can be unstable, one has to check the stability of the Sarma mean field solutions using the curvature criterion, which says that the second derivative of thermodynamic potential with respect to the gap needs to be positive. It is known that the curvature criterion correctly discards the unstable solutions, but the metastable solutions may still survive. This may cause only minor quantitative changes in the first-order phase transition [13].

We shall consider a weak coupling limit $U=2.64J$, where the BS equation in the GRPA provides a good approximation for the collective-mode energies. The strength of the SOC is $\lambda =0.1J$, and the density of the majority and minority components are $f_\uparrow =0.275$ and $f_\downarrow =0.225$, respectively (the polarization is $P=(f_\uparrow -f_\downarrow )/(f_\uparrow +f_\downarrow )=0.1$). The corresponding tight-binding form of the electron energy is $\xi _{\uparrow ,\downarrow }(\mathbf k )=2J(1-\cos k_x)+2J(1-\cos k_y)-\mu _{\uparrow ,\downarrow }$, where $\mu _{\uparrow ,\downarrow }$ is the chemical potential (the lattice constant $a=1$).

In the mean field approximation the single-particle spectrum is

$$\begin{aligned} \Omega (\mathbf k )=\pm \sqrt{S(\mathbf k )+\frac{1}{2}\left( \omega ^2_+(\mathbf k )+\omega ^2_-(\mathbf k )\right) + 2A(\mathbf k )},\\ \omega (\mathbf k )=\pm \sqrt{S(\mathbf k )+\frac{1}{2}\left( \omega ^2_+(\mathbf k )+\omega ^2_-(\mathbf k )\right) - 2A(\mathbf k )},\\ A(\mathbf k )=\sqrt{\left( S(\mathbf k ) +\eta ^2(\mathbf k )\right) \left( \chi ^2(\mathbf k )+\Delta ^2\right) -S(\mathbf k )\Delta ^2}, \end{aligned}$$

where $\Delta$ is the gap, $S(\mathbf k )=4\lambda ^2\left( \sin ^2(k_y)+\cos ^2(k_x)\right)$, $\chi (\mathbf k )=(\xi _{\uparrow }(\mathbf k )+\xi _{\downarrow }(\mathbf k ))/2$, $\eta (\mathbf k )=(\xi _{\uparrow }(\mathbf k )-\xi _{\downarrow }(\mathbf k ))/2+h_{z}$, and $\omega _{\pm }(\mathbf k )=\sqrt{\chi ^2(\mathbf k )+\Delta ^2}\pm \eta (\mathbf k )$. By minimizing the Helmholtz free energy

$$\begin{aligned}&F=\frac{\Delta ^2}{U}+\sum _\mathbf{k }\left[ \chi (\mathbf k )-\frac{1}{2}\left( \Omega (\mathbf k )+ \omega (\mathbf k )\right) \right] \nonumber \\ {}&-T\sum _\mathbf{k }\left[ \ln \left( 1+e^{-\Omega (\mathbf k )/T}\right) +\ln \left( 1+e^{-\omega (\mathbf k )/T}\right) \right] +f_\uparrow \mu _{\uparrow }+f_\downarrow \mu _{\downarrow } \end{aligned}$$

(4)

with respect to $\mu _{\uparrow }$, $\mu _{\downarrow }$ and $\Delta$, one can obtained the mean field number and gap equations for the chemical potentials and the gap:

$$\begin{aligned} n= & {} f_\uparrow +f_\downarrow =1-\sum _\mathbf{k }\left[ \frac{1}{2}-f(\Omega (\mathbf k ))\right] \frac{\chi (\mathbf k )}{\Omega (\mathbf k )} \left( 1+\frac{S(\mathbf k )+\eta ^2(\mathbf k )}{A(\mathbf k )}\right) \nonumber \\&-\sum _\mathbf{k }\left[ \frac{1}{2}-f(\omega (\mathbf k ))\right] \frac{\chi (\mathbf k )}{\omega (\mathbf k )} \left( 1-\frac{S(\mathbf k )+\eta ^2(\mathbf k )}{A(\mathbf k )}\right) , \end{aligned}$$

(5)

$$\begin{aligned} nP= & {} f_\uparrow -f_\downarrow =-\sum _\mathbf{k }\left[ \frac{1}{2}-f(\Omega (\mathbf k ))\right] \frac{\eta (\mathbf k )}{\Omega (\mathbf k )} \left( 1+\frac{\chi ^2(\mathbf k )+\Delta _0^2}{A(\mathbf k )}\right) \nonumber \\&-\sum _\mathbf{k }\left[ \frac{1}{2}-f(\omega (\mathbf k ))\right] \frac{\eta (\mathbf k )}{\omega (\mathbf k )} \left( 1-\frac{\chi ^2(\mathbf k )+\Delta _0^2}{A(\mathbf k )}\right) , \end{aligned}$$

(6)

$$\begin{aligned} \frac{1}{U}= & {} \sum _\mathbf{k }\left[ \frac{1}{2}-f(\Omega (\mathbf k ))\right] \frac{1}{2\Omega (\mathbf k )} \left( 1+\frac{\eta ^2(\mathbf k )}{A(\mathbf k )}\right) \nonumber \\&+\sum _\mathbf{k }\left[ \frac{1}{2}-f(\omega (\mathbf k ))\right] \frac{1}{2\omega (\mathbf k )} \left( 1-\frac{\eta ^2(\mathbf k )}{A(\mathbf k )}\right) . \end{aligned}$$

(7)

Here, $f(\omega )$ is the Fermi–Dirac distribution function.

In what follows, we calculate the mean field single-particle spectrum and the dispersion of the collective excitations at zero temperature. First, we shall assume that there is no out-of-plane Zeeman field. When $h_{z}=0$, the solutions of the number and the gap equations at zero temperature provide for the chemical potentials $\mu _\uparrow =2.857J$, $\mu _\downarrow =2.186J$, and the gap $\Delta =0.266J$. With these results, we have checked the stability of the Sarma mean field solutions, and it is found that the Sarma phase is stable.

It is known, that within the Gaussian approximation the collective-mode dispersion is defined by the following $2 \times 2$ secular determinant [8, 9]:

$$\begin{aligned}&\displaystyle Z_{2\times 2}(\omega ,\mathbf Q )=\left| \begin{array}{c@{\quad }c} z_{11} &{}z_{12} \\ z_{21} &{} z_{22} \\ \end{array} \right| ,\\&\displaystyle z_{11}= 1+\frac{U}{2}\left( K_{2233}(\omega ,\mathbf Q )+K_{1144}(\omega ,\mathbf Q )-K_{1234}(\omega ,\mathbf Q )-K_{2143}(\omega ,\mathbf Q )\right) , \\&\displaystyle z_{12}= \frac{U}{2}\left( K_{1414}(\omega ,\mathbf Q )+K_{2323}(\omega ,\mathbf Q )-K_{2413}(\omega ,\mathbf Q )-K_{1324}(\omega ,\mathbf Q )\right) , \\&\displaystyle z_{21}=\frac{U}{2}\left( K_{1414}(\omega ,\mathbf Q )+K_{2323}(\omega ,\mathbf Q )-K_{4231}(\omega ,\mathbf Q )-K_{3142}(\omega ,\mathbf Q )\right) , \\&\displaystyle z_{22}= 1+\frac{U}{2}\left( K_{3322}(\omega ,\mathbf Q )+K_{4411}(\omega ,\mathbf Q )-K_{3412}(\omega ,\mathbf Q )-K_{4321}(\omega ,\mathbf Q )\right) . \end{aligned}$$

The two-particle free propagators $K_{ijkl}(\omega ,\mathbf Q )$ are defined in the Appendix. The collective-mode dispersion $\omega (\mathbf Q )$ is obtained by the condition that the secular determinant becomes equal to zero.

Instead of integrating out the fermion fields, one can transform the quartic terms to a quadratic forms by introducing a four-component boson field which mediates the interaction of fermions [14, 15]. This approach is similar to the situation in quantum electrodynamics, where the photons mediate the interaction of electric charges. This similarity allows us to apply the powerful functional integral technique to derive the Dyson equation and the BS equation for the poles of the single-particle and two-particle Green’s functions, respectively. In the case of vanishing spin–orbit coupling, this approach provides an $8\times 8$ secular determinant [16]. As shown in the Appendix, the existence of the SOC term in the Hamiltonian leads to more complicated $12\times 12$ secular determinant $M_{12\times 12}(\omega ,\mathbf Q )$.

In Fig. 1, we plot the collective-mode excitation energy $\omega (Q_x)$ as a function of the wave vector $\mathbf Q =(Q_x,0)$ along the x-axis, calculated by the Gaussian approximation (triangles) and by the BS approach in the GRPA (circles). The speed of sound, provided by the Gaussian approximation, is $u =$ 1.66 $Ja/\hbar$, while the speed of sound, calculated within the BS approach is $u =$ 1.35 $Ja/\hbar$. Thus, the Gaussian approximation overestimates the speed of sound by about 23 %. More interestingly, the dispersion curve calculated from the BS equation clearly shows the existence of a roton-like minimum, while there is no such a minimum within the Gaussian approximation. The corresponding roton gap is $\Delta _{r} = 0.2025 J$ and the critical flow velocity, obtained around the roton minimum from the BS equation, is $v_\mathrm{c} =$ 0.51 $Ja/\hbar$. The two dispersion curves are remarkably different around the roton minimum; instead of the expected roton-like structure, the dispersion curve provided by the Gaussian approximation monotonically increases in this region. At higher wave vectors, $Q_x >$ 0.16 $\pi /a$, the two dispersion curves have essentially the same behavior.

Fig. 1

Collective modes dispersion $\omega (Q_x)$ of an imbalanced SOC Fermi gas in a 2D square optical lattice along the $(Q_x,0)$ direction, obtained by the Gaussian approximation (triangles) and the Bethe–Salpeter equation in the GRPA (circles)

Our results suggest that the Gaussian approximation overestimates the speed of sound of the Goldstone mode, and fails to reproduced the roton-like structure of the collective-mode dispersion which appears after the linear part of the dispersion. The question naturally arises here, whether the Gaussian approximation still can be used to estimate the speed of sound if one takes into account not only the SOC, but Zeeman fields as well. This question will be answered in the next Section, where we investigate the speed of sound of a balanced superfluid Fermi gas in a two-dimensional square optical lattice with a Rashba spin–orbit coupling (in the xy plane) and an out-of-plane Zeeman field.

Fig. 2

The chemical potential (diamonds) and the gap (squares) of a balanced superfluid Fermi gas in a 2D square optical lattice as a function of the out-of-plane Zeeman field. The system parameters are: filling factor $f=0.5$, the attractive interaction $U=5.2J$, and the strength of the Rashba spin–orbit coupling $\lambda =0.1J$

3 Speed of Sound Near the Transition from the Gapped Superfluid Phase to the Topological Phase

We consider a balanced system with a filling factor $f=0.5$. Since we intend to calculate the speed of sound near the transition from the gapped superfluid phase to the topological phase in the Gaussian and in the BS approximations, we increase the attractive interaction to $U=5.2J$ (the stronger the strength of the interaction, the greater are the differences between the speed of sound in the Gaussian and in the BS approximations). The strength of the Rashba spin–orbit coupling is $\lambda =0.1J$, and there exists an out-of-plane Zeeman field $h_{z}$. In such a system a phase transition can be accessed by varying the Zeeman field. The transition from the gapped superfluid phase to the topological phase is characterized by the quasiparticle excitation gap that closes at $h_\mathrm{c}=\sqrt{\mu ^2+\Delta ^2}$ and reopens with increasing $h_{z}>h_\mathrm{c}$.

In Fig. 2, we present the chemical potential (diamonds) and the gap (squares) for different Zeeman fields, obtained by solving numerically the number and the gap equations. The critical field $h_\mathrm{c}$ marks the phase transition between the topologically nontrivial (negative side) and the topologically trivial (positive side) superfluid phases. As can be seen, during this transition, the gap $\Delta$ is still finite even though the quasiparticle excitation gap is closed. This suggests that there is a quantum phase transition separating the parameter regimes $h_{z}<h_\mathrm{c}$ and $h_{z}>h_\mathrm{c}$, even though the system in both regimes is an s-wave superfluid.

In Fig. 3, we have plotted the speed of sound along x direction ($\mathbf Q =(Q_x,0)$) as a function of the Zeeman field, calculated within the Gaussian approximation and from the BS equation in the GRPA. As can be seen, close to the phase transition boundary the speed of sound calculated within the two approaches is essentially the same. The inset in the figure shows that the minimum of the speed of sound is located at the phase transition boundary $h_\mathrm{c}$. The same behavior was previously found by applying the Gaussian approximation in the case of a 2D superfluid atomic Fermi gas with Rashba type spin-orbit coupling and an out-of-plane Zeeman field [6, 8], and in the case of a 3D FF type of superfluid Fermi gas with Rashba spin-orbit coupling [9] (in the xy plane) and two Zeeman fields [in-plane (hx) and out-of-plane (hz)].Thus, our calculations are in agreement with the suggestion made in [9], that by measuring the minimum of the speed of sound one can unambiguously detect the topological phase transition boundary.

Fig. 3

The speed of sound along the x direction as a function of the Zeeman field, calculated within the Gaussian approximation (circles) and by the BS approach (triangles). The values of the chemical potential and gap are shown in Fig. 2

4 Summary

In summary, we have derived the BS equation in the GRPA for the collective excitation energy of a Fermi gas in a 2D square lattice with an attractive contact interaction, assuming the existence of a nonvanishing Rashba SOC and an out-of-plane Zeeman field. We have calculated the collective-mode dispersion within the Gaussian approximation, and from the BS equation assuming the existence of a Sarma superfluid state. It is found that the Gaussian approximation (i) overestimates the speed of sound of the Goldstone mode, and (ii) fails to reproduce the roton-like structure of the collective-mode dispersion which appears in the BS approach.

We also have investigated the speed of sound of a balanced spin–orbit-coupled atomic Fermi gas near the boundary of the topological phase transition driven by an out-of-plane Zeeman field. It is shown that the minimum of the speed of sound is located at the topological phase transition boundary, and this fact can be used to confirm the existence of a topological phase transition.

Finally, it is worth mentioning that our $12\times 12$ secular determinant can be used to describe collective excitations of superfluid states of Cooper pairs with nonzero momentum. These states could occur in the population-imbalanced case between a fermion with momentum $\mathbf k + \mathbf q$ and spin $\uparrow$ and a fermion with momentum $-\mathbf k + \mathbf q$, and spin $\downarrow$ . As a result, the pair momentum is $2\mathbf q$. A finite pairing momentum implies a position-dependent phase of the order parameter, which in the Fulde–Ferrell case varies as a single plane wave $\Delta _{i_1,i_2}\equiv \Delta \mathbf q e^{\imath 2\mathbf q.r _{i_1}}\delta (\mathbf r _{i_1}-\mathbf r _{i_2})$. The only difference is in the mean field single-particle Green’s functions used to obtain the two-particle free propagator $K_{ijkl}(\omega ,\mathbf Q )$. The poles of the mean field single-particle Green’s function in the present of SOC and the Zeeman field are defined by very long expressions, and therefore, the numerical solution of the mean field set of two number equations, the gap equation and the equation for the Fulde–Ferrell vector $\mathbf q$ is an ambitious task which will be left as a subject of our future research.

Appendix: Bethe–Salpeter Approach to the Collective Modes of a Spin–Orbit-Coupled Superfluid Fermi Gas

We start with introducing a four-component boson field $A_{\alpha }(z)$ interacting with fermion fields $\widehat{\overline{\psi }} (y)=\widehat{\Psi }^\dag (y)/\sqrt{2}$ and $\widehat{\psi }(x)=\widehat{\Psi }(x)/\sqrt{2}$ ($\alpha =1,2,3,4$, $\{x,y\}=(\mathbf r _i,u)$, $z=(\mathbf r _i,v)$, $0\le u,v \le \beta =(k_BT)^{-1}$, and $k_B$ is the Boltzmann constant), where

$$\begin{aligned} \widehat{\Psi }(x)=\left( \begin{array}{c} \psi _\uparrow (x) \\ \psi _\downarrow (x) \\ \psi ^\dag _\uparrow (x)\\ \psi ^\dag _\downarrow (x)\\ \end{array} \right) ,\quad \widehat{\Psi }^\dag (y)=\left( \psi ^\dag _\uparrow (y)\psi ^\dag _\downarrow (y) \psi _\uparrow (y)\psi _\downarrow (y) \right) . \end{aligned}$$

The generalized single-particle Green’s function $\widehat{G}(x_1;y_2)=-{\big \langle }{\widehat{T}_u\left( \widehat{\Psi }(x_1)\otimes \widehat{\overline{\Psi }}(y_2)\right) }{\big \rangle }$ is represented by a $4\times 4$ matrix, which includes all possible thermodynamic averages (the symbol $\otimes$ means a tensor product between two matrices). As in quantum electrodynamics, where the photons mediate the interaction of electric charges, we define an action of the following form $S= S^{(F)}_0+S^{(B)}_0+S^{(F-B)}$, where $S^{(F)}_0=\widehat{\overline{\psi } }(y)\widehat{G}^{(0)-1}(y;x)\widehat{\psi } (x)$, $S^{(B)}_0=\frac{1}{2}A_{\alpha }(z)D^{(0)-1}_{\alpha \beta }(z,z')A_{\beta }(z')$, $S^{(F-B)}=\widehat{\overline{\psi }} (y)\widehat{\Gamma }^{(0)}_{\alpha }(y,x\mid z)\widehat{\psi } (x)A_{\alpha }(z)$. The action $S^{(F)}_0$ describes the fermion part of the system. The generalized inverse Green’s function of free fermions $\widehat{G}^{(0)-1}(y;x)$ is given by the following $4\times 4$ matrix:

$$\begin{aligned} \widehat{G}^{(0)-1}(y;x)=\sum _\mathbf{k ,\omega _m}\exp \left[ \imath \mathbf k. (\mathbf r _i-\mathbf r _{i'})-\omega _m(u-u')\right] \widehat{G}^{(0)-1}(\mathbf k ,\imath \omega _m) , \end{aligned}$$

where the symbol $\sum _{\omega _m}$ is used to denote $\beta ^{-1}\sum _{m}$ (for fermion fields $\omega _m= (2\pi /\beta )(m +1/2) ;m=0, \pm 1, \pm 2,...$). In the case of the population-imbalanced Fermi gas with a Rashba SO coupling and an out-of-plane Zeeman field, the noninteracting Green’s function is

$$\begin{aligned}&\displaystyle \widehat{G}^{(0)-1}(\mathbf k ,\imath \omega _m)=\left( \begin{array}{c@{\quad }c} g^{(0)-1}_{11} &{}0 \\ 0 &{} g^{(0)-1}_{22} \\ \end{array} \right) , \\&\displaystyle g^{(0)-1}_{11}=\left( \begin{array}{c@{\quad }c} \imath \omega _m-\xi _\uparrow (\mathbf k )-h_{z}&{}-2\lambda \left( \sin k_x+\imath \sin k_y\right) \\ -2\lambda \left( \sin k_x-\imath \sin k_y\right) &{}\imath \omega _m-\xi _\downarrow (\mathbf k )+h_{z} \end{array} \right) , \\&\displaystyle g^{(0)-1}_{22}=\left( \begin{array}{c@{\quad }c} \imath \omega _m+\xi _\uparrow (\mathbf k )+h_{z}&{}-2\lambda \left( \sin k_x-\imath \sin k_y\right) \\ -2\lambda \left( \sin k_x+\imath \sin k_y\right) &{}\imath \omega _m+\xi _\downarrow (\mathbf k )-h_{z} \end{array} \right) , \end{aligned}$$

The action $S^{(B)}_0$ describes the boson field which mediates the fermion–fermion on-site interaction in the Hubbard Hamiltonian. The bare boson propagator in $S^{(B)}_0$ is defined as

$$\begin{aligned} \widehat{D}^{(0)} (z,z')=\delta (v-v')U\delta {j,j'}\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} 0&{}1&{}0&{}0 \\ 1 &{}0 &{}0&{}0 \\ 0 &{}0 &{}0&{}0\\ 0 &{}0 &{} 0&{}0 \end{array} \right) . \end{aligned}$$

The Fourier transform of this boson propagator is given by

$$\begin{aligned} \widehat{D}^{(0)} (z,z')=\frac{1}{N} \sum _\mathbf k \sum _{\omega _p} e^{\left\{ \imath \left[ \mathbf k. \left( \mathbf r _j-\mathbf r _{j'}\right) -\omega _p\left( v-v'\right) \right] \right\} }\widehat{D}^{(0)}(\mathbf k ), \end{aligned}$$

where

$$\begin{aligned} \widehat{D}^{(0)}(\mathbf k )= \left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} 0&{}U &{}0&{}0 \\ U &{}0 &{}0&{}0 \\ 0 &{}0 &{}0&{}0 \\ 0 &{}0 &{} 0&{}0 \end{array} \right) . \end{aligned}$$

The interaction between the fermion and the boson fields is described by the action $S^{(F-B)}$. The bare vertex $\widehat{\Gamma }^{(0)}_{\alpha }(y_1;x_2\mid z)=\widehat{\Gamma }^{(0)}_{\alpha }(i_1,u_1;i_2, u_2\mid j,v)=\delta (u_1-u_2)\delta (u_1-v)\delta _{i_1i_2}\delta _{i_1j}\widehat{\Gamma }^{(0)}(\alpha )$ is a $4\times 4$ matrix, where

$$\begin{aligned} \widehat{\Gamma }^{(0)}(\alpha )=\frac{1}{2}(\gamma _0+\alpha _z)\delta _{\alpha 1} +\frac{1}{2}(\gamma _0-\alpha _z)\delta _{\alpha 2}+ \frac{1}{2}(\alpha _x+\imath \alpha _y)\delta _{\alpha 3}+ \frac{1}{2}(\alpha _x-\imath \alpha _y)\delta _{\alpha 4}. \end{aligned}$$

The Dirac matrix $\gamma _0$ and the matrices $\widehat{\alpha }_i$ are defined as (when a four-dimensional space is used, the electron spin operators $\sigma _i$ has to be replaced by $\widehat{\alpha }_i \gamma _0$)

$$\begin{aligned} \gamma _0=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} 1&{}0&{}0&{}0 \\ 0&{}1&{}0&{}0 \\ 0&{} 0&{} -1&{}0 \\ 0&{} 0&{} 0&{}-1 \\ \end{array} \right) ,\quad \widehat{\alpha }_i=\left( \begin{array}{c@{\quad }c} \sigma _i &{} 0 \\ 0&{} \sigma _y\sigma _i\sigma _y \\ \end{array} \right) , i=x,y,z. \end{aligned}$$

The relation between the Hubbard model and our model system can be demonstrated by applying the Hubbard–Stratonovich transformation for the fermion operators:

$$\begin{aligned}&\int D\mu [A]\exp \left[ \widehat{\overline{\psi }} (y)\widehat{\Gamma }^{(0)}_{\alpha }(y;x|z)\widehat{\psi }(x)A_{\alpha }(z)\right] \\&\quad = \exp \left[ -\frac{1}{2}\widehat{\overline{\psi }} (y)\widehat{\Gamma }^{(0)}_{\alpha }(y;x|z)\widehat{\psi }(x) D_{\alpha ,\beta }^{(0)}(z,z') \widehat{\overline{\psi }} (y')\widehat{\Gamma }^{(0)}_{\beta }(y';x'|z')\widehat{\psi }(x')\right] . \end{aligned}$$

The functional measure $D\mu [A]$ is chosen to be

$$\begin{aligned} D\mu [A]=DAe^{-\frac{1}{2}A_{\alpha }(z)D_{\alpha ,\beta }^{(0)-1}(z,z') A_{\beta }(z')},\int \mu [A] =1. \end{aligned}$$

By following the standard steps in the functional integral technique, one can derive the Dyson equation $G^{-1}=G^{(0)-1}-\Sigma$, and the BS equation $[K^{-1}-I]\Psi =0$ for the poles of the single-particle Green’s function, G, and the poles of the two-particle Green’s function, respectively. Here, $G^{(0)}$ is the free single-particle propagator, $\Sigma$ is the fermion self-energy, I is the BS kernel, and the two-particle free propagator $K = GG$ is a product of two fully dressed single-particle Green’s functions. The kernel of the BS equation is defined as a sum of the direct interaction, $I_\mathrm{d}=\delta \Sigma ^{F}/ \delta G$, and the exchange interaction $I_{\mathrm{exc}}=\delta \Sigma ^{H}/ \delta G$, where $\Sigma ^{F}$ and $\Sigma ^{H}$ are the Fock and the Hartree parts of the fermion self-energy $\Sigma$. Since the fermion self-energy depends on the two-particle Green’s function, the positions of both poles must to be obtained by solving the Dyson and BS equations self-consistently.

In practice, the GRPA allows us to decouple the Dyson equations from the BS equation. According to this approximation, one can use the mean field approximation for the single-particle Green’s function:

$$\begin{aligned} \widehat{G}_{MF}^{-1}(\mathbf k ,\imath \omega _m)=\left( \begin{array}{cc} g^{(0)-1}_{11} &{}\imath \Delta \sigma _y \\ -\imath \Delta \sigma _y &{} g^{(0)-1}_{22} \\ \end{array} \right) . \end{aligned}$$

In the GRPA the direct interaction in the BS kernel is calculated by a linearized Fock term and exact Hartree term:

$$\begin{aligned}&\Sigma _0^F(i_1,u_1;i_2,u_2)_{n_1n_2}\\&\quad =-U\delta _{i_1,i_2}\delta (u_1-u_2)\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} 0 &{}G_{12}(1;2)&{}0&{}-G_{14}(1;2) \\ G_{21}(1;2)&{} 0 &{}-G_{23}(1;2)&{}0 \\ 0 &{} -G_{32}(1;2)&{}0&{}G_{34}(1;2) \\ -G_{41}(1;2) &{} 0&{}G_{43}(1;2)&{}0 \\ \end{array} \right) , \end{aligned}$$

$$\begin{aligned}&\widehat{\Sigma }^H(i_1,u_1;i_2,u_2)\\&\quad =\frac{U}{2}\delta _{i_1,i_2}\delta (u_1-u_2)\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} G_{22}-G_{44} &{}0&{}0&{}0 \\ 0&{} G_{11}-G_{33} &{}0&{}0 \\ 0 &{} 0&{}G_{44}-G_{22}&{}0 \\ 0&{} 0&{}0&{}G_{33}-G_{11} \\ \end{array} \right) . \end{aligned}$$

In the GRPA the BS equation for the sixteen BS amplitude $\Psi ^\mathbf{Q }_{n_2,n_1},\quad \{n_1,n_2\}=1,2,3,4$ is

$$\begin{aligned}\Psi ^\mathbf{Q }_{n_2n_1}=K\left( \begin{array}{c@{\quad }c} n_1 &{} n_3 \\ n_2 &{} n_4 \\ \end{array} |\omega (\mathbf Q )\right) \left[ I_\mathrm{d}\left( \begin{array}{c@{\quad }c} n_3 &{} n_5 \\ n_4 &{} n_6 \\ \end{array} \right) +I_\mathrm{exc}\left( \begin{array}{c@{\quad }c} n_3 &{} n_5 \\ n_4 &{} n_6 \\ \end{array} \right) \right] \Psi ^\mathbf{Q }_{n_6,n_5}, \end{aligned}$$

where $I_\mathrm{d}\left( \begin{array}{c@{\quad }c} n_1 &{} n_3 \\ n_2 &{} n_4 \\ \end{array} \right) =-\Gamma ^{(0)}_\alpha (n_1,n_3)D^{(0)}_{\alpha \beta } \Gamma ^{(0)}_\beta (n_4,n_2)$ and $I_\mathrm{exc}\left( \begin{array}{c@{\quad }c} n_1 &{} n_3 \\ n_2 &{} n_4 \\ \end{array} \right) = \frac{1}{2}\Gamma ^{(0)}_\alpha (n_1,n_2)D^{(0)}_{\alpha \beta } \Gamma ^{(0)}_\beta (n_4,n_3)$ are the direct and the exchange interactions [16], correspondingly. The two-particle free propagator K in the GRPA is defined as follows:

$$\begin{aligned} K\left( \begin{array}{c@{\quad }c} n_1 &{} n_3 \\ n_2 &{} n_4 \\ \end{array} |\omega (\mathbf Q )\right) =\int \frac{d\Omega }{2\pi } \int \frac{d^d\mathbf k }{(2\pi )^2}G^{MF}_{n_1n_3} \left( \mathbf k +\mathbf Q ,\Omega +\omega (\mathbf Q )\right) G^{MF}_{n_4n_2}(\mathbf k ,\Omega ). \end{aligned}$$

In the mean field approximation, the zero-temperature single-particle Green’s function is a $4\times 4$ matrix which has components are

$$\begin{aligned}&G^{MF}_{n_1n_2}(\mathbf k ,\omega )=\nonumber \\ {}&\frac{A_{n_1n_2}(\mathbf k )}{\omega -\Omega (\mathbf k )+\imath 0^+}+\frac{B_{n_1n_2}(\mathbf k )}{\omega +\Omega (\mathbf k )-\imath 0^+}+\frac{C_{n_1n_2}(\mathbf k )}{\omega -\omega (\mathbf k )+\imath 0^+}+\frac{D_{n_1n_2}(\mathbf k )}{\omega +\omega (\mathbf k )-\imath 0^+}. \end{aligned}$$

The corresponding expressions for $A_{n_1n_2}(\mathbf k ),B_{n_1n_2}(\mathbf k ),C_{n_1n_2}(\mathbf k )$, and $D_{n_1n_2}(\mathbf k )$ can be obtained by inverting the $\widehat{G}_{MF}^{-1}(\mathbf k ,\imath \omega _m)$ matrix.

The above BS equation can be rewritten in matrix form as $\left( \widehat{I}+U\widehat{M}\right) \widehat{\Psi }=0$, where the $16\times 16$ matrix $\widehat{I}+U\widehat{M}$ can be simplify to a $12\times 12$ matrix. Thus, the collective-mode dispersion $\omega (\mathbf Q )$ in the GRPA is obtained by the condition $Det |M_{12\times 12}(\omega ,\mathbf Q )|=0$, where the matrix $M_{12\times 12}$ is defined as

$$\begin{aligned}&\displaystyle M_{12\times 12}(\omega ,\mathbf Q )=\left( \begin{array}{c@{\quad }c@{\quad }c} M_{11} &{}M_{12}&{}M_{13} \\ M_{21} &{}M_{22}&{}M_{23} \\ M_{31} &{}M_{32}&{}M_{33} \\ \end{array} \right) , \\&\displaystyle M_{11}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} 1+U/2\left( K_{1221}-K_{1441}\right) &{}-UK_{1211}&{}UK_{1411}&{}-UK_{1121}\\ U/2\left( K_{2221}-K_{2441}\right) &{}1-UK_{2211}&{}UK_{2411}&{}-UK_{2121} \\ U/2\left( K_{4221}-K_{4441}\right) &{}-UK_{4211}&{}1+UK_{4411}&{}-UK_{4121} \\ U/2\left( K_{1222}-K_{1442}\right) &{}-UK_{1212}&{}UK_{1412}&{}1-UK_{1122}\\ \end{array} \right) , \\&\displaystyle M_{12}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} U/2\left( K_{1111}-K_{1331}\right) &{}UK_{1321}&{}UK_{1231}&{}U/2\left( K_{1441}-K_{1221}\right) \\ U/2\left( K_{2111}-K_{2331}\right) &{}UK_{2321}&{}UK_{2231}&{}U/2\left( K_{2441}-K_{2221}\right) \\ U/2\left( K_{4111}-K_{4331}\right) &{}UK_{4321}&{}UK_{4231}&{}U/2\left( K_{4441}-K_{4221}\right) \\ U/2\left( K_{1112}-K_{1332}\right) &{}UK_{1322}&{}UK_{1232}&{}U/2\left( K_{1442}-K_{1222}\right) \\ \end{array} \right) , \\&\displaystyle M_{13}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} -UK_{1431}&{}UK_{1141}&{}-UK_{1341}&{}U/2\left( K_{1331}-K_{1111}\right) \\ -UK_{2431}&{}UK_{2141}&{}-UK_{2341}&{}U/2\left( K_{2331}-K_{2111}\right) \\ -UK_{4431}&{}UK_{4141}&{}-UK_{4341}&{}U/2\left( K_{4331}-K_{4111}\right) \\ -UK_{1432}&{}UK_{1142}&{}-UK_{1342}&{}U/2\left( K_{1332}-K_{1112}\right) \\ \end{array} \right) , \\&\displaystyle M_{21}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} U/2\left( K_{1222}-K_{1442}\right) &{}-UK_{2212}&{}UK_{2412}&{}-UK_{2122} \\ U/2\left( K_{3222}-K_{3442}\right) &{}-UK_{3212}&{}UK_{3412}&{}-UK_{3122} \\ U/2\left( K_{2223}-K_{2443}\right) &{}-UK_{2213}&{}UK_{2413}&{}-UK_{2123}\\ U/2\left( K_{3223}-K_{3443}\right) &{}UK_{3213}&{}UK_{3413}&{}-UK_{3123}\\ \end{array} \right) , \end{aligned}$$

$$\begin{aligned} M_{22}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} 1+U/2\left( K_{2112}-K_{2332}\right) &{}UK_{2322}&{}UK_{2232}&{}U/2\left( K_{2442}-K_{2222}\right) \\ U/2\left( K_{3112}-K_{3332}\right) &{}1+UK_{3322}&{}UK_{3232}&{}U/2\left( K_{3442}-K_{3222}\right) \\ U/2\left( K_{2113}-K_{2333}\right) &{}UK_{2323}&{}1+UK_{2233}&{}U/2\left( K_{2443}-K_{2223}\right) \\ U/2\left( K_{3113}-K_{3333}\right) &{}UK_{3323}&{}UK_{3233}&{}1+U/2\left( K_{3443}-K_{3223}\right) \\ \end{array} \right) , \end{aligned}$$

$$\begin{aligned}&\displaystyle M_{23}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} -UK_{2432}&{}UK_{2142}&{}-UK_{2342}&{}U/2\left( K_{2332}-K_{2112}\right) \\ -UK_{3432}&{}UK_{3142}&{}-UK_{3342}&{}U/2\left( K_{3332}-K_{3112}\right) \\ -UK_{2433}&{}UK_{2143}&{}-UK_{2343}&{}U/2\left( K_{2333}-K_{2113}\right) \\ -UK_{3433}&{}UK_{3143}&{}-UK_{3343}&{}U/2\left( K_{3333}-K_{3113}\right) \\ \end{array} \right) , \\&\displaystyle M_{31}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} U/2\left( K_{4223}-K_{4443}\right) &{}-UK_{4213}&{}UK_{4413}&{}-UK_{4123}\\ U/2\left( K_{1224}-K_{1444}\right) &{}-UK_{1214}&{}UK_{1414}&{}-UK_{1124}\\ U/2\left( K_{3224}-K_{3444}\right) &{}-UK_{3214}&{}UK_{3414}&{}-UK_{3124}\\ U/2\left( K_{4224}-K_{4444}\right) &{}-UK_{4214}&{}UK_{4414}&{}-UK_{4124}\\ \end{array} \right) , \\&\displaystyle M_{32}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} U/2\left( K_{4113}-K_{4333}\right) &{}UK_{4323}&{}UK_{4233}&{}U/2\left( K_{4443}-K_{4223}\right) \\ U/2\left( K_{1114}-K_{1334}\right) &{}UK_{1324}&{}UK_{1234}&{}U/2\left( K_{1444}-K_{1224}\right) \\ U/2\left( K_{3114}-K_{3334}\right) &{}UK_{3324}&{}UK_{3234}&{}U/2\left( K_{3444}-K_{3224}\right) \\ U/2\left( K_{4114}-K_{4334}\right) &{}UK_{4324}&{}UK_{4234}&{}U/2\left( K_{4444}-K_{4224}\right) \\ \end{array} \right) , \\&\displaystyle M_{33}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} 1-UK_{4433}&{}UK_{4143}&{}-UK_{4343}&{}U/2\left( K_{4333}-K_{4113}\right) \\ -UK_{1434}&{}1+UK_{1144}&{}-UK_{1344}&{}U/2\left( K_{1334}-K_{1114}\right) \\ -UK_{3434}&{}UK_{3144}&{}1-UK_{3344}&{}U/2\left( K_{3334}-K_{3114}\right) \\ -UK_{4434}&{}UK_{4144}&{}-UK_{4344}&{}1+U/2\left( K_{4334}-K_{4114}\right) \\ \end{array} \right) . \end{aligned}$$

Here, we have used a short notation $K\left( \begin{array}{c@{\quad }c} n_1 &{} n_3 \\ n_2 &{} n_4 \\ \end{array} |\omega (\mathbf Q )\right) \equiv K_{n_1n_3n_4n_2}$.