1 Introduction

The optical nonlocal soliton, a spatially localized optical beam in a nonlocal nonlinear medium, has been investigated substantially both theoretically and experimentally [1,2,3,4,5,6,7,8,9,10,11]. The nonlocal nonlinear optical materials are generally classified into three distinct categories: weakly nonlocal, generally nonlocal, and highly nonlocal [1,2,3,4,5,6,7,8,9,10,11]. The relative length of the beam width and the characteristics length of the nonlinear media’s response function determine the nonlocality nomenclature of optical materials [1,2,3,4,5,6,7]. The nonlocal solitons have been achieved in all three distinct categories of nonlocal nonlinearity; weakly nonlocal solitons [3, 4], generally nonlocal solitons [6] and highly nonlocal solitons [5]. Theoretically, in general, two types of profiles have been used for the response functions for the nonlocality: the Gaussian-type profile [1], and the exponential-decay type profile [2]. The soliton in the strongly nonlocal nonlinearity experiences a number of fascinating phenomena, such as a large phase shift [5], attraction dynamics between out-of-phase solitons [2, 8, 12, 13], attraction dynamics between dark solitons [14], and long-range interaction between solitons [15]. Nonlocal nonlinearity distinguishes these events from those with local nonlinearity. An optical beam passing through a highly nonlocal medium will undergo periodic changes [16]. The highly nonlocal wave system will eventually self-organize into a spatially confined incoherent solitonic structure, which will be fundamentally distinct from incoherent optical solitons. Nematicons are an unique type of nonlocal solitons that may be found in nematic liquid crystals (NLC) [17, 18]. This makes them part of a specific category of nonlocal soliton. It has been shown that anisotropic nonlocal vector solitons can exist in unbiased NLC [19].

Different kinds of beam profiles have been employed to stimulate nonlocal soliton. To mention just a few, these include the Gaussian profile [5, 12], super-Gaussian profile [20], the Hermite–Gaussian (HG) profile [21], Laguerre–Gaussian profile [7], cosh–Gaussian profile [22], Ince–Gaussian profile [23, 24], complex-variable-function Gaussian profile [25], and variable sinh-Gaussian profile [26]. Dai et. al. showed the excitation of a Hermite-Gaussian spatiotemporal soliton in a (3+1)-dimensional partly nonlocal nonlinear system [27]. Yang et al. confirmed that a competitive nonlocal nonlinear system that obeys the parabolic law can play host to dark and single optical solitons [28]. Also, a nonlocal medium that exhibits orientational nonlinearity has the potential to give rise to a stable vortex soliton [29].

A substantial amount of research is dedicated to the study of shock waves in nonlocal media. For instance, the formation of optical spatial shock waves in nonlocal nonlinear media [30] and generation of giant collective incoherent shock waves from coherent shocklets in nonlocal turbulent flows [31].

Clusters of solitons of various types, namely, Gaussian solitons, multi-pole solitons, and nested solitons, have been excited in a strongly nonlocal (3+1)-dimensional in-homogeneous parity-time (PT) symmetric nonlinear system [32]. A novel integrable nonlocal nonlinear Schrödinger equation that has a Lax pair and an unlimited number of conservative quantities has been proposed in a PT-symmetric system [33]. Hu et al. [34] showed that the presence of nonlocality has the potential to expand the stability zone of defect solitons in PT-symmetric optical lattices. In addition to this, nonlocal nonlinearity has a significant impact on the interaction dynamics of the Airy beam solitons. In the presence of strong nonlocal nonlinearity, two in phase Airy solitons develop a long-range attractive force between each other and may form a stable bound state consisting of in phase as well as out of phase breathing Airy solitons [35]. It may be noted here that both the in phase and the out of phase airy solitons normally repel each other in local nonlinear media.

The modulation instability (MI) in the nonlocal nonlinear medium is a highly sought-after phenomenon [1, 7] among researchers. MI has been studied in a (1+1)-dimensional cubic-quintic nonlocal nonlinear (CQNNL) system [36]. These CQNNL systems contain solitons with varied profile configurations. For instance, an elliptic soliton may be stimulated in an anisotropic (1+2) dimensional CQNNL system [37], or a vortex soliton can be created and stabilized in a (1+2) dimensional CQNNL system [38]. Mishra at el. demonstrated a bright soliton generation and bifurcation in a (1+1) dimensional CQNNL system [10, 39,40,41]. The experimental confirmation of a single nonlocal spatial soliton has been established [42, 43]. Further, the interaction between a nonlocal soliton pair has been demonstrated [8, 15], and MI of such solitons has been portrayed [44]. The study of dark and singular optical solitons is carried out within the framework of parabolic law nonlocal nonlinearities [28]. The soliton turbulence self-organization of a non-integrable system, which develops as a result of the MI, collapses when it is subjected to the impact of extremely nonlocal nonlinearity [45].

Another significant consequence of nonlocal nonlinearity is the MI with long-range gravitational interaction. Gravity is inherently nonlinear and long-range (i.e., nonlocal) interaction. Thus, a nonlocal nonlinearity provides a natural framework for exploring the long-range general relativity effects. This encourages drawing analogies between gravitational and optical phenomena, e.g., optical analogs of the Newton–Schrodinger equation, gravitational attraction, and light-trapping in the wake of optical solitons (pls see Ref. [46], op. cit.). A vast literature is available in the framework of the Newton–Schrödinger or Schrödinger–Poisson equation presenting the gravitational interactions [46,47,48]. A nonlinear optical setup can be introduced to illustrate gravitational dynamics that require a highly nonlocal nonlinearity [47]. Hidden coherent soliton states are found from a modulationally unstable initial condition in the framework of the Schrödinger–Poisson or Newton–Schrödinger equations (please see ref. [48], op. cit.). They are hidden means fully immersed in random wave fluctuations in the gravitational incoherent structures. Given gravity’s nonlinear and nonlocal nature, these hidden coherent solitons might be detectable in nonlocal nonlinear optics experiments or dipolar Bose–Einstein condensates.

The nonlocal material for narrow beams can possess higher-order-diffraction (HD), which primarily includes third-order-diffraction (TD) and fourth-order-diffraction (FD) in addition to second-order-diffraction (SD). The TD and FD have a significant impact on the propagation dynamics of narrow beams, making it an essential topic for researchers to anticipate some unique and fascinating phenomena [49]. The effect of FD has been studied in solitons in the semi-infinite gap of a PT-symmetric periodic potential [50], fundamental and dipole gap solitons in PT-symmetric mixed linear-nonlinear optical lattices [51], spatial solitons and stability with parity-time-symmetric potentials [52], and soliton with parity-time symmetric potentials [53]. Recently, multi-pole solitons in nonlocal nonlinear media with FD [54] have also been investigated. Both bright and dark type pure-quartic solitons in weakly nonlocality [55] do not possess oscillating tails, which is different from the conventional pure-quartic solitons.

Although the effects of TD and FD are significant in the study of solitons, they have received relatively little scientific attention in the context of nonlocal nonlinear media. As such their effects on beam dynamics have not been compared. Therefore, the present study aims to generate a nonlocal soliton in nonlocal nonlinear media by incorporating TD and FD. The study emphasizes on demonstrating the effects of TD, FD, and the characteristic length of the response function on soliton beam propagation and MI along with linear stability analysis. The structure of this article is as follows: In Sect. 2, the mathematical modeling of the system and its variational analysis are described. Section 3 presents the creation of solitons and their dynamics that have been determined analytically by the Lagrangian variational method (LVM) and numerically by split-step Fourier method (SSFM). Section 4 demonstrates the linear stability analysis of the nonlocal nonlinear solitons, while Sect. 5 portrays the MI of the solitons. The article has been concluded in Sect. 6.

2 Mathematical modeling

The dynamics of an optical beam propagating through a nonlocal nonlinear medium incorporating the SD, TD, and FD can be represented by the following nonlocal nonlinear Schrödinger equation (NNLSE),

$$\begin{aligned}&i\, \frac{\partial U}{\partial z} +\beta _2\, \frac{\partial ^2 U}{\partial x^2} +i\,\beta _3\, \frac{\partial ^3 U}{\partial x^3} + \beta _4\, \frac{\partial ^4 U}{\partial x^4} +i\,\epsilon U \nonumber \\&\quad +\rho \, U \int _{-\infty }^\infty R(x-\xi )\,|U(\xi ,z)|^2 d\xi =0 . \end{aligned}$$
(1)

Here, U(xz) is the slowly varying spatial part of the paraxial beam profile with z as the longitudinal propagation distance and x as the transverse coordinate. \(|U(\xi ,Z)|^2\) represents the intensity of the beam. The parameters \(\beta _2\), \(\beta _3\), and \(\beta _4\) are the coefficients of SD, TD, and FD, respectively. \(\beta _2={1}/({2\,k})\), \(\rho =k\,\eta\), where k is the wave number and \(\eta\) is a constant specified by the material property. Both \(\beta _3\), and \(\beta _4\) can be a fraction of \(\beta _2\). \(\epsilon\) captures the attenuation in the system. R(x) is the Gaussian nonlocal response kernel of the isotropic Kerr nonlinear medium, such that \(\int R(x) dx=1\;\). We choose, \(R(x) =({1}/{\sqrt{\pi }\sigma }) \exp \left( {-{x^2}/{\sigma ^2}}\right)\) where \(\sigma\) is the characteristic length of the response function (or the extent of nonlocality). \(\rho\) is the coefficient of the nonlocal nonlinearity. The class of nonlocality is determined, for all intents and purposes, by the characteristic length (\(\sigma\)). When the beam width is much smaller than the characteristic length (\(w<<\sigma\)), it induces a highly or strongly nonlocal nonlinearity. On the other hand, when the beam width is approximately equal to the characteristic length (\(w \approx \sigma\)), it results in weakly nonlocal nonlinearity. The general case of nonlocality falls somewhere between these two extremes, with a range extending from weak nonlocality to strong nonlocality [1,2,3,4,5,6,7]. Indeed, (\(w>>\sigma\)) indicates local nonlinearity. Typically, nonlocal nonlinear optical media is characterized by an exponential-shaped response function, as described in Eq. (35) of Ref. [1], due to its association with thermal nonlocal nonlinearity. However, in our current case, it’s worth noting that we are utilizing a Gaussian-shaped nonlinear response function.

The effects of higher-order diffraction can be understood by employing the unidirectional Helmholtz propagation equation [56], wherein the expansion of the dispersion/diffraction relation gives rise to the higher-order diffraction terms; but only of even orders. In contrast, the present investigation considers the third-order diffraction term. The motivation is twofold. Firstly, this theoretical work foreruns experimental realization and shows what characteristics the solitons may show if materials having both even and odd order diffractions and nonlocal nonlinearity are used. The recent advancement in metamaterial research unveiled the potential of many unique, unconventional materials (e.g., metamaterial) and so for the medium modeled here. Moreover, the model can help in identifying the potential materials, and consequently guiding experimental efforts.

The NNLSE, i.e.,  (Eq. 1) is non-integrable. We adopt approximate analytical methods as well as numerical methods. Following the analytical approach  (Eq. 1) has been solved through the Lagrangian variational method (LVM) as mentioned in [5, 20, 57,58,59]. The LVM approach is a popular approximation technique proposed by Anderson in 1983 [60]. The effectiveness of this method is reliant on the selection of an appropriate trial function or ansatz. We begin the present analysis with the following Gaussian ansatz:

$$\begin{aligned} U\left( {z,x}\right)&= \sqrt{\frac{E_0}{\sqrt{\pi }w(z)}}\, \exp \left( - {\frac{(x-x_0(z))^2}{2\,w^2(z)}} \right) \,\nonumber \\&\qquad \exp \left[ -i\,\left( \frac{c(z)(x-x_0(z))^2}{2\,w^2(z)} + \Omega (z)(x-x_0(z))\right. \right. \nonumber \\&\quad \left. \left. -\phi (z)\right) \right] , \end{aligned}$$
(2)

where \(E_0\) represents the initial beam energy of the profile (i.e., (Eq. 2)), while w(z) denotes the beam width. c(z), \(x_0(z)\), \(\Omega (z)\) and \(\phi (z)\) are, respectively, the phase front curvature, position of beam center, nonlinear frequency shift, and longitudinal phase of the beam. In the next step of the LVM technique, a Lagrangian density L corresponding to (Eq. 1) is determined such that \(\delta L/\delta \psi ^*=\delta L/\delta \psi =0\) reproduces (Eq. 1). In this case, we have

$$\begin{aligned} L=&\frac{i}{2} \left( U^* \frac{\partial U}{\partial z}-U \frac{\partial U^*}{\partial z}\right) - \beta _2 \left|{\frac{\partial U}{\partial x}}\right|^2 \nonumber \\&\quad +i\,\beta _3 \left( \frac{\partial U^*}{\partial x} \frac{\partial ^2U}{\partial x ^2}-\frac{\partial U}{\partial x} \frac{\partial ^2U^*}{\partial x ^2}\right) +\beta _4 \left( \frac{\partial ^2U^*}{\partial x ^2}\frac{\partial ^2U}{\partial x ^2}\right) \nonumber \\&+ \frac{1}{2}\,{\rho \,\textrm{e}^{-2\,\epsilon \,z}}\, |U|^2\int ^\infty _{-\infty } R(x-\xi ) \, \left( {|U(\xi ,z)|^2} \right) \, d\xi \end{aligned}$$
(3)

Following the conventional formulation of the LVM technique, the (Eq. 2) is now put into (Eq. 3) to obtain the reduced Lagrangian (not shown here). Upon integration over the entire available space we get the total Lagrangian \(\langle L \rangle\) as

$$\begin{aligned} \langle L \rangle&=-\frac{E_0\,}{4\,w^2}\, \left( 2\,\beta _2 S_1+4\,{w}^{2}\Omega \,{\frac{\partial x_0}{\partial z}}+4\,{w}^{2} \frac{\partial \phi }{\partial z}-{w}^{2}{\frac{\partial c}{\partial z}}\right. \nonumber \\&\left. +4\,\beta _2\,{w}^{2}{\Omega }^{2}+2\,wc\,{\frac{\partial w}{\partial z}} \right) +{\frac{{\beta _3}\,E_0\, \Omega \, S_2 }{{w}^{2}}} +{\frac{{\textrm{e}^{-2\,\epsilon \,z}}\rho \,{E_0}^{2}}{2\,\sqrt{\pi }\sqrt{2\,{w}^{2}+{\sigma }^{2}}}} \nonumber \\&+{\frac{{\beta _4}\,E_0\, \left( 3+12\,{\Omega }^{2}{w}^{2}+4 \,{\Omega }^{4}{w}^{4}+6\,{c}^{2}+3\,{c}^{4}+12\,{w}^{2}{c} ^{2}{\Omega }^{2} \right) }{4\,{w}^{4}}}. \end{aligned}$$
(4)

Here \(S_1=\left( 1+{c}^{2}+2\,{\Omega }^{2}{w}^ {2} \right)\) and \(S_2=\left( 3+3\,{c}^{2}+2\,{\Omega }^{2}{w}^{2} \right)\). The total Lagrangian is varied with respect to the free beam parameters \(r_j = c(z), w(z), x_0(z), \Omega (z)\) and \(\phi (z)\) using the following Euler–Lagrange equation,

$$\begin{aligned} \frac{\partial \langle L \rangle }{\partial r_j} - \frac{d}{dz} \left( {\frac{\partial \langle L \rangle }{\partial \dot{r}_j}}\right) =0, \end{aligned}$$
(5)

to obtain the following set of coupled first-order ordinary differential equations (ODEs) that illustrates the evolution of the beam parameters during propagation.

$$\begin{aligned} \frac{\partial c}{\partial z}&= \frac{(-2\,\beta _2 +12\,\beta _3\,\Omega )(1+c^2)}{w^2}+ {\frac{ 6\,{\beta _4}\, \left( 1+{c}^{2} \right) S_1 }{{w}^{4}}} \nonumber \\&\quad +{\frac{2\,{w}^{2}\rho \,{E_0}}{\sqrt{ \pi } \left( 2\,{w}^{2}+{\sigma }^{2} \right) ^{3/2} \left( {\textrm{e}^{ \epsilon \,z}} \right) ^{2}}} \end{aligned}$$
(6)
$$\begin{aligned} \frac{\partial x_0}{\partial z}&= -2\,\beta _2\Omega +\frac{3\,{\beta _3\, S_1 }}{{w}^{2}} +{\frac{2\,\beta _4\Omega \, S_2 }{{w}^{2}}} \end{aligned}$$
(7)
$$\begin{aligned} \frac{\partial w}{\partial z}&= -{\frac{2\,\beta _2\,c}{w}}+{\frac{12\,\beta _3\,c\,\Omega }{w}}+{\frac{ 6\,\beta _4 c\, S_1 }{{w}^{3}} },\end{aligned}$$
(8)
$$\begin{aligned} \frac{\partial \Omega }{\partial z}&= 0. \end{aligned}$$
(9)

Further differentiation of (Eq. 8) with respect to z yields

$$\begin{aligned}&{\frac{{ 1}}{\beta _2}}\frac{\partial ^2y}{\partial z ^2}= {\frac{4\,\rho \,E_0\, \left( S_3\,{w_0}^{2}{y}^{2}+6\, \beta _4\,{c}^{2} \right) }{\beta _2\,\sqrt{\pi }{\left( 2\,{w_0}^{2}{y}^{2}+{\sigma }^{2} \right) ^{3/2}{\textrm{e}^{ 2\,\epsilon \,z}}}\,y}}+4\, \left( S_3\,{w_0}^{2}{y}^{2} \right. \nonumber \\&\qquad \left. +3\,\beta _4\,\left( 1+{c}^{2} \right) \right) {\frac{\left( S_3\,{w_0}^{2}{y}^{2}-3\,\beta _4 \,{c}^{2} \left( 1+{c}^{2} \right) \right) }{{w_0}^{6}\beta _2\,{y}^{ 7}}} \equiv F(y). \end{aligned}$$
(10)

Here,\(y(z)=w/w_0\), with \(w_0=w(0)\). \(S_3=-\beta _2+6\,\beta _3\,\Omega +6\,\beta _4\,{\Omega }^{2}\). Under the influence of the force F(y), Eq. 10 behaves in a manner akin to the nonlinear harmonic oscillator equation with unit mass. This force F(y) ensures that the effects of the diffractive and refractive forces remain in a state of equilibrium. The critical power for soliton propagation can be found by assuming that the two opposing forces are in perfect equilibrium with one another and \(y=1\). The critical power for soliton propagation can be written as

$$\begin{aligned} E_c= \left( \frac{\sqrt{ \pi } (1+c^2)\left( 2\,{w}^{2}+{\sigma }^{2} \right) ^{3/2}}{\rho }\right) \left( \frac{\,\beta _2 -6\,\beta _3\,\Omega }{w^4}+{\frac{3\,{ \beta _4}\, S_1 }{{w}^{6}}}\right) . \end{aligned}$$
(11)

The formation of optical soliton is largely reliant on the critical beam power \(E_c\).

3 Beam dynamics

Before delving into the beam propagation characteristics in SD, TD, and FD media, it is necessary to provide some examples of such media. SD media are abundant as they constitute the basic diffraction mechanism. Birefringent materials can exhibit TD, as in the case of some examples [61]. Certain materials, such as photorefractive liquid crystal materials [62, 63], can demonstrate both TD and FD. Additionally, metamaterials can exhibit both TD [64] and FD [65], depending on their structure. Nonlinear optical materials, like potassium dihydrogen phosphate, can also exhibit both TD and FD [66, 67]. Alongside the material properties, the type of diffraction exhibited by media is also dependent on the frequency, intensity, and polarization of the incident light. In this investigation, we provide a generic study suitable for media with HD. Throughout this investigation, the parameters are set to \(w=1.0,\,c=0.0,\,\rho =1/6,\, \Omega =0.0,\, \beta _2=0.5,\,\beta _3=-0.1,\,\text {and}\, \beta _4=0.05\,\), unless specified otherwise. It’s worth emphasizing that in the case of conventional materials (that follow the regular dispersion/diffraction relations discussed in [56]), the fourth-order diffraction coefficient is typically determined by the second-order diffraction coefficient and the laser wavelength. In such cases, the fourth-order diffraction coefficient is essentially a predetermined fraction of the second-order diffraction coefficient. Nevertheless, the present study takes different values of the fourth-order diffraction coefficient in some instances to tackle scenarios that encompass materials (such as those exhibiting PT symmetries), and advanced technologies that necessitate a more comprehensive modeling approach, extending beyond the conventional dispersion/diffraction relations.

Equation 11 shows the dependence of the critical beam energy (\(E_c\)) on the characteristic length of the response function (\(\sigma\)). \(E_c\) increases nonlinearly with an increase in \(\sigma\) (Fig. 1a). SD Media alone or in combination with TD media show almost similar variation, however, the rate of increment gets slower in the presence of FD (Fig. 1a). Thus, the presence of FD lowers the critical beam energy for soliton formation.

Fig. 1
figure 1

a The variation of critical energy of soliton formation with the characteristic length (\(\sigma\)). b The variation of potential V(y) with respect to y. The black dotted line is for SD, the cyan color line is for SD+TD and the blue color line is for SD+TD+FD

As F(y) is a conservative force, the relation \(F(y)=-dV(y)/dy\) determines the corresponding potential V(y) as

$$\begin{aligned} V&\left( y \right) = \frac{24\,\beta _4\,{c}^{2}\rho \,E_0\, }{ {\beta _2 \sqrt{\pi }}{ {\sigma }^{2}{{\sqrt{{\sigma }^{2}}}} {\textrm{e}^{2\,\epsilon \,z}} }}\left[ \ln \left( {\frac{2\,{ \sigma }^{2}+2\,\sqrt{{\sigma }^{2}}\sqrt{2\,{w_0}^{2}{y}^{2}+{ \sigma }^{2}}}{y}} \right) \right. -\nonumber \\&\left. \ln \left( 2\,{\sigma }^{2}+2\,\sqrt{{ \sigma }^{2}}\sqrt{2\,{w_0}^{2}+{\sigma }^{2}} \right) \right] + {\frac{2\,\rho \,E_0\, \left( {\sigma }^{2}S_3-12\,\beta _4{c}^{2} \right) }{\beta _2\,\sqrt{\pi } {\sigma }^{2} {\textrm{e}^{2\,\epsilon \,z}} }} \nonumber \\&\left( \frac{1}{\sqrt{2\,{w_0}^{2}{y}^{2}+{\sigma }^{2}}} -\frac{1}{\sqrt{2\,{w_0}^{2}+{\sigma }^{2}}} \right) + {\frac{2\,S_3^2\,(1-y^2)}{\beta _2\,{w_0}^{2}{y}^{2}}}\nonumber \\&-{\frac{3\,\beta _4\, \left( {c}^{4}-1 \right) S_3(1-y^4)}{\beta _2\,{w_0}^{4}{y}^{4}}} -{\frac{6\,{c}^{2}{\beta _4}^{2} \left( 1+{c}^{2} \right) ^{2}(1-y^6)}{\beta _2\, {w_0}^{6}{y}^{6}}} \end{aligned}$$
(12)

The potential profile V(y) is a parabolic one for all types of diffracting media. Alike in the case of critical energy, the profiles of SD with or without TD are the same and become significantly wider in the presence of FD media (Fig. 1b).

Now Eqs. (6)–(9) (derived by LVM) are solved numerically using Runge–Kutta fourth-order (RK-4) algorithm [68] to determine the evolution of beam parameters with propagation distance (z). When the initial energy \(E_0\) of a beam is exactly equal to \(E_c\) (Eq. 11) the beam propagates with constant width (Fig. 2c), whereas when the initial energy is set lower (Fig. 2a) or higher (Fig. 2e) than critical energy, the beam width oscillates almost periodically. Anyways, in all the cases soliton formation is witnessed. Now, in the presence of loss in the system the beam continues to oscillate periodically for all energy levels but spreads eventually (right column of Fig. 2). To highlight the effect of the characteristics length \(\sigma\) on oscillating soliton width we plotted the soliton amplitude for all nonlocal cases, namely highly nonlocal (\(\sigma =10.0\)), generally nonlocal (\(\sigma =4.0\)), weakly nonlocal (\(\sigma =1.0\)) and local (\(\sigma =0.1\)), with \(E_0\) set to \(10\%\) higher than \(E_c\) (i.e., \(E_0=1.1\times E_c\)) for various combination of diffractions (Fig. 3 and Fig. S1). A comparison of the propagation dynamics with SD, TD, and FD for different characteristic lengths \(\sigma\) (Fig. 3) demonstrates that a decrease in the characteristic lengths leads to an increase in soliton amplitude and a decrease in the frequency of oscillation. Also, the presence of HD of any kind decreases the frequency of oscillation and increases the soliton amplitude (Fig. S1a–d).

Fig. 2
figure 2

The variation of beam width (w(z)) with propagation distance (z) for \(\sigma =10.0\). a \(E_0<E_c\), c \(E_0=E_c\) and e \(E_0>E_c\). The first column represents non-dissipative system, while the second column is the dissipative case corresponding to the first column

Fig. 3
figure 3

The variation of beam width (w(z)) with propagation distance (z) for a SD, b SD+TD, c SD+FD, and d SD+TD+FD. The black, cyan, blue, and magenta lines are for the characteristic length \(\sigma =10.0,\,\sigma =4.0,\,\sigma =1.0,\) and \(\sigma =0.1\), respectively

Different order of HD affects the beam center (\(x_0\)) differently. The presence of SD and FD makes no shift of the beam center (Fig. 4a, c, e, g), but TD pushes the beam center to the negative side as the beam propagates (Fig. 4b, d) and it is more prominent for the dissipative system. This large shift of the beam center (Fig. 4h) can be utilized for developing all-optical sensor devices.

At this point, it is useful to validate the analytical results obtained from LVM. We adopt the split-step Fourier method (SSFM) [59, 69, 70] for the direct numerical solution of Eq. (1) using an initial beam profile similar to what is given in Eq. (2). This pseudo-spectral method is faster than the Finite-Difference Time-Domain (FDTD) method while doing negligible compromise with the results. For a range of system parameters, we obtained the solitonic beam propagation. Examples of such soliton beams are given in Fig. 5 for different combinations of various orders of diffraction. A comparison of such numerically (SSFM) obtained solitons (Fig. 5) with the analytically (LVM) obtained ones (Fig. 2) reveals that both types of solitons exhibit a periodic change in amplitudes; however, the oscillation in amplitude is a bit lesser in the former cases. Moreover, the presence of either TD (Fig. 5b) or FD (Fig. 5c) along with SD creates snake-like soliton propagation, while a combination of SD, TD, and FD suppresses such dynamics (Fig. 5d). Again, a decrease in \(\sigma\) clearly increases the periodicity or in other words decreases the frequency of the snaking-like oscillation (Fig. 5c, e, f). Such snake-like propagation is not vivid in analytical results (Fig 2). In essence, the numerical method reveals a more vivid picture, while the analytical method provides the idea of initial values for the numerical simulations. Furthermore, soliton behavior involving third-order dispersion in the temporal domain is commonly associated with the emission of the dispersive wave. However, no such emission is prominently observed with third-order diffraction in the spatial domain, at least for the set of system parameters used in the current investigation. Please note that Fig S1 and all other supplementary figures are available in the supplementary file.

Fig. 4
figure 4

The dislocation or shift of beam center (\(x_0(z)\)) with propagation distance (z). a, e are for SD, b, f are for SD+TD, c, g are for SD+FD, and d, h are for SD+TD+FD. The first row is for the non-dissipative system and the second row is for the dissipative system (\(\epsilon =0.2\))

Fig. 5
figure 5

The variation of beam amplitude (A(z)) with propagation distance (z) for \(\sigma =10.0\). a SD, b SD+TD, c SD+FD, d SD+TD+FD, e SD+FD for \(\sigma =4.0\), and f SD+FD for \(\sigma =1.0\)

Fig. 6
figure 6

Unstable focus phase plots (clockwise outward spirals) for \(\Lambda _+\) and \(\Lambda _{-}\) are complex conjugates. a \(c=-5\times 10^{-5}\) and b \(c=+5\times 10^{-5}\). c stable center phase plots for \(\sigma =10\) and \(c=0\)

4 Linear stability analysis

Fig. 7
figure 7

Real and imaginary component of the Eigenvalues in the parametric characteristic length (\(\sigma\)) - chirp (c) zone for only SD. The first row represents \(\Lambda _+\), and the second row represents \(\Lambda _{-}\). The real Eigenvalue is in the first column, whereas the imaginary Eigenvalue is in the second column

Fig. 8
figure 8

Real and imaginary component of the Eigenvalues in the parametric characteristic length (\(\sigma\)) - chirp (c) zone for all SD, TD & FD. The first row represents \(\Lambda _+\), and the second row represents \(\Lambda _{-}\). The real Eigenvalue is in the first column, whereas the imaginary Eigenvalue is in the second

Fig. 9
figure 9

The variation of beam amplitude (A(z)) with beam width w(z) for \(\sigma =10.0\). a SD, b SD+TD, c SD+FD, d SD+TD+FD, e SD+FD for \(\sigma =4.0\), and f SD+FD for \(\sigma =1.0\)

The acceptability of the solutions obtained by the LVM technique (preceding section) is contingent on the degree to which they are stable against perturbations. Therefore, a linear stability analysis [71,72,73] is carried out in the vicinity of the equilibrium points (\(c_0, w_0\)) obtained by LVM. firstly, we introduce a negligibly small perturbation \(c^\star\) and \(w^\star\) to the equilibrium points. The new initial conditions are \(c=c_0+c^\star\) and \(w=w_0+w^\star\). After linearizing Eqs. (6) and (8) around a stationary point, we get a pair of equations, which can be expressed in a matrix form as

$$\begin{aligned} \frac{d}{dz} \begin{pmatrix} c^\star \\ w^\star \end{pmatrix} = \begin{pmatrix} f_{11} &{} f_{12} \\ f_{21} &{} f_{22} \end{pmatrix} \begin{pmatrix} c^\star \\ w^\star \end{pmatrix} \end{aligned}$$
(13)

with,

$$\begin{aligned} f_{11}&= {\frac{4\,c\,S_3}{{w}^{2}}}+{\frac{24\,\beta _4\,c\, \left( {c}^{2}+ 1 \right) }{{w}^{4}}}, \end{aligned}$$
(14)
$$\begin{aligned} f_{12}&= {\frac{4\, \left( \beta _2-6\,\beta _3\,\Omega +6\,\beta _4\,{\Omega }^{2} \right) \left( 1+{c}^{2} \right) }{{w}^{3}}}\nonumber \\&\quad -{\frac{24\,\beta _4\, \left( 1+{c}^{2} \right) S_1 }{{w}^{5}}} -{\frac{4\,w\rho \,E_0\, \left( w^2-\sigma ^2 \right) }{\sqrt{\pi } \left( 2\,{w}^{2}+{\sigma }^{2} \right) ^{5/2} {\textrm{e}^{2\,\epsilon \,z}} }},\end{aligned}$$
(15)
$$\begin{aligned} f_{21}&= {\frac{2S_3}{w}}+{ \frac{6\,\beta _4 (1+3\,{c}^{2})}{{w}^{3}}},\end{aligned}$$
(16)
$$\begin{aligned} f_{22}&= -{\frac{2\,c S_3}{{w}^{2}}}-{\frac{18\beta _4\,c \left( 1+{c}^{2}\right) }{{w}^{4}}}. \end{aligned}$$
(17)

This results in the following characteristic Eigenvalue equation:

$$\begin{aligned} {\Lambda ^2 - (f_{11}+f_{22})\Lambda +(f_{11}f_{22}-f_{12}f_{21})}=0. \end{aligned}$$
(18)

The two Eigenvalues (\(\Lambda _+\) and \(\Lambda _{-}\)) of the \(2\times 2\) stability matrix (right hand side of the Eq. 13) can be calculated using the following formula,

$$\begin{aligned} \Lambda _\pm&= \frac{1}{2} (f_{11}+f_{22}) \pm \frac{1}{2} \sqrt{f_{11}^2 - 2 f_{11} f_{22}+ f_{22}^2+4\,f_{12}f_{21}} \end{aligned}$$
(19)

The type or class of equilibrium points can be determined by the characteristics of the Eigenvalues (\(\Lambda _+\) and \(\Lambda _{-}\)). For example, we fixed the values of system parameters as \(w=1,\, \beta _2=0.5,\, \epsilon =0,\, \rho =1/6,\, \Omega =0.0,\, \beta _3=-0.1,\,\) and \(\beta _4=0.05\,\) while varying the values of \(\sigma\) and c to determine the various kinds of equilibrium points as per Ref. [71]. A choice \(\sigma =10\) and \(c=+0.33\) results in negative Eigenvalues (i.e., \(\Lambda _+< \Lambda _{-} < 0\)) while \(\sigma =10\) and \(c=-0.33\) results in positive Eigenvalues (i.e., \(\Lambda _+> \Lambda _{-} > 0\)); both are the condition for a stable node or star. Now, for \(\sigma =10\) and \(c=\pm 0.000005\) the Eigenvalues are complex conjugate, which is the condition of stable or unstable focus (Fig. 6a, b). When \(\sigma =10\) and \(c=0\), the Eigenvalues become pure imaginary and the equilibrium point becomes a center (Fig. 6c). Similarly, \(\sigma =1\) or 0.5 or 0.1 and \(c=-1.53\) leads to \(\Lambda _+< 0 < \Lambda _{-}\), satisfying the saddle point equilibrium criterion. By setting \(c=\pm 0.35\) or \(c=\pm 0.91\), either of the Eigenvalues become zero (i.e., \(\Lambda _{\pm }=0\) and \(\Lambda _{\mp }\ne 0\)) that produce a degenerate type equilibrium point. By selecting the proper quadrants of the real and imaginary plots of the Eigenvalues, the parametric region for various kinds of equilibrium points may be identified as shown in Fig 7 for SD only (Fig. S3 is for SD & TD), and Fig 8 is for the combination of SD, TD, & FD.

The stability of the solitons discussed till now can be confirmed by conducting direct numerical simulations of the original NLS, Eq. (1). As an illustration, the beam amplitude versus beam width phase diagrams (Fig. 9) obtained solely through numerical methods (specifically, SSFM) reveal well-confined trajectories during propagation. These trajectories signify stable soliton beam dynamics for all combinations of diffractions and various \(\sigma\) values.

5 Modulation instability

MI is a ubiquitous phenomenon that manifests in most nonlinear systems in nature and also for optical beams [69, 74]. It causes small amplitude and phase perturbations, arising from noise, to quickly amplify under the combined influences of nonlinearity and diffraction (or dispersion, in the temporal domain). Consequently, a wide optical beam (or a quasi-CW pulse) tends to disintegrate while propagating, leading to either filamentation (or break-up into pulse trains) or extinction [75]. The study of nonparaxial MI was conducted in reference [56], initially focusing on a modified Nonlinear Schrödinger Equation (NLS) with higher-order diffraction terms (i.e., the corrections to the linear term). Subsequently, the investigation extended to include nonlinear term corrections, introducing minor nonlocal interactions. Filament generation and the impact of the applied corrections have been studied in the zone beyond the linear phase of MI. It is found that nonparaxial conditions lead to less pronounced filament focusing, mainly due to Fourier components cut off in Fourier space, restricting focusing in real space. In the current study, we are focusing on different orders of diffraction and the linear stability of solitons. Therefore, it is relevant to conduct a thorough investigation of modulation instability (MI), which usually occurs in the same parameter region where soliton formation takes place. A comprehensive review of MI of solitons in spatially nonlocal nonlinear media, including the formation, interaction, and collapse of spatial solitons, can be found in Ref. [76]. To examine the MI, let’s assume the NNLSE (Eq. 1) has a plane wave solution of the form \(U(z,x)= \sqrt{ p_0} \,\exp \left( {i\left( k_0x-\omega _0 z \right) }\right) \,\), where \(p_0\) is the incident power. Here, the parameters \(\omega _0,\, \beta _2,\, k_0,\, \beta _3,\,\beta _4,\, \epsilon\) and \(\rho\) are related by the equation \(\omega _0 = \beta _2 k_0^2 - \beta _3 k_0^3 - \beta _4 k_0^4-i\epsilon -\rho \, p_0\,.\) Let us now perform a linear stability analysis on the plane wave solutions by considering a small modulation in the wave solution as

$$\begin{aligned} U(z,x)=\left( \sqrt{ p_0} + a_1(z,x) \right) \,\exp \left( {i\left( k_0x-\omega _0 z \right) }\right) . \end{aligned}$$
(20)

Here, \(a_1(z,x)\) denotes a small complex perturbation. By substituting (Eq. 20) into the NNLSE (Eq. 1), and linearizing around the solution (Eq. 20), we obtain th perturbed evolution equation as

$$\begin{aligned}&i\, \frac{\partial a_1}{\partial \tau } +\Theta _1 \frac{\partial ^2 a_1}{\partial \xi ^2} +i\,\Theta _2\, \frac{\partial ^3 a_1}{\partial \xi ^3} + \beta _4\, \frac{\partial ^4 a_1}{\partial \xi ^4} \nonumber \\&\quad +2\rho \,p_0 \int _{-\infty }^\infty Re(a_1) R(\xi -\xi ^\prime ) \; d\xi ^\prime =0, \end{aligned}$$
(21)

where \(\Theta _1=\left( \beta _2- 3\,k_0\beta _3 - 6\,k_0^2\beta _4 \right)\) and \(\Theta _2=(\beta _3+4\,k_0\beta _4)\). We utilized the change of variable as \(\tau =z\) and \(\xi =x-\left( 2\beta _2 k_0 + 3i\,k_0^2\beta _3-4k_0^3\beta _4 \right) z\) to arrive at the above equation. Two coupled equations can be generated by separating the perturbation into imaginary and real components (i.e., \(a_1=u+iv\)) as:

$$\begin{aligned} \frac{\partial {u}}{\partial \tau }&= - \Theta _1 {\frac{\partial ^{2}v}{\partial {\xi }^{2}}} -\Theta _2\,{\frac{\partial ^{3}u}{\partial {\xi }^{3}}} -\beta _4\,{\frac{\partial ^{4}v}{\partial {\xi }^{4}}} \end{aligned}$$
(22)
$$\begin{aligned} \frac{\partial {v}}{\partial \tau }&= \Theta _1 {\frac{\partial ^{2}u}{\partial {\xi } ^{2}}}- \Theta _2\,{\frac{\partial ^{3}v}{\partial {\xi }^{3}}} +\beta _4\,{\frac{\partial ^{4}u}{\partial {\xi }^{4}}} \nonumber \\&\quad +2\,\rho \,p_0\,\int _{-\infty }^\infty u(\xi ^\prime ,\tau ) R(\xi -\xi ^\prime ) \; d\xi ^\prime \,. \end{aligned}$$
(23)

Here the quantities \({\hat{u}}, {\hat{v}}\) and \({\hat{R}}\) are defined by the Fourier transforms,

$$\begin{aligned} \hat{\Psi _i}(k,\tau ) = \int _{-\infty }^{\infty } \Psi _i(\xi ,\tau )\,\exp (ik\xi )\;d\xi , \nonumber \\ \end{aligned}$$
(24)

where \(\Psi _i= u,v,R\). The linearized system is transformed to a set of ordinary differential equations in k space using the convolution theorem for Fourier transformations, as follows:

$$\begin{aligned} \frac{\partial {\hat{u}}}{\partial \tau }&= -Q {\hat{v}}, \end{aligned}$$
(25)
$$\begin{aligned} \frac{\partial {\hat{v}}}{\partial \tau }&= \left( Q+2\,\rho \,p_0 R({\hat{k}}) \right) {\hat{u}}. \end{aligned}$$
(26)

Here \(Q= 11\,\beta _4 k_0^4 + 4\,\beta _3 k_0^3-\beta _2 k_0^2\). This can be expressed in matrix form as \(\partial _\tau X = AX,\) where the vector X and matrix A are defined as

$$\begin{aligned} X=\begin{bmatrix} {\hat{u}}\\ {\hat{v}} \end{bmatrix},\qquad \qquad A=\begin{bmatrix} 0 &{} -Q \\ Q+2\,\rho \,p_0\, R({\hat{k}}) &{} 0 \end{bmatrix} \end{aligned}$$
(27)

The eigenvalues \(\lambda\) of the matrix A are given by

$$\begin{aligned} \lambda ^2+Q^2+2\,Q\,\rho \, p_0\, R({\hat{k}})=0. \end{aligned}$$
(28)

According to Ref. [74], \(R({\hat{k}}) = 1\) for local media, \(R({\hat{k}})=1-\gamma \,k^2\) for weakly nonlocal nonlinear media with \(\gamma\) being the diffusion parameter, and \(R({\hat{k}})=exp(-\sigma ^2\,k^2/4)\) for generally nonlocal nonlinear media. The growth rate of the MI is defined as the real of the Eigenvalues \(\lambda\) (i.e., \(\Re (\lambda )\)), and can be obtained from Eq. (28) as

$$\begin{aligned} |\Re (\lambda )| = \Re \left( \sqrt{-{Q}^{2}-2\,Q\rho \,p_0\,R({\hat{k}})}\right) . \end{aligned}$$
(29)

The growth rate can be significantly modified by tuning different system parameters and order of diffraction. We first take the case of weakly nonlocal nonlinear media (i.e., \(R({\hat{k}})=1-\gamma \,k^2\)) to study the growth rate variation with wave number \(k_0\). It is observed that the growth rate of the sidebands more or less increases with an increase in incident beam power \(p_0\) at a different pace for different orders and combinations of diffraction (Fig. 10). SD and SD+FD lead to symmetric sidebands (Fig. 10a, c), while the presence of TD makes the upper sidebands more amplified; eventually leaving the side bands asymmetric (Fig. 10b, d). The sidebands get more amplified for higher values of the diffusion parameter. This is prominent for a media with SD (Fig. 11a) and higher-order diffraction too follows the same trend (Fig. 11b–d). Figure S4 in the supplementary material is the same as Fig. 11 except the X-axis is chosen a little wider. The variation of the growth rate profile with respect to \(p_0\) corresponding to the Fig. 11a is given in Fig. 12. Also the variation of the growth rate profile with respect to \(p_0\) corresponding to the Fig. 11b–d are given at supplementary material in Figs. S5, S6, and S7, respectively.

Unlike in the weakly nonlocal nonlinear media, the sidebands get more amplified at lower values of \(\sigma\) for various order diffraction and their combinations (Fig. 13) in the generally nonlocal nonlinear media (i.e., \(R({\hat{k}})=exp(-\sigma ^2\,k^2/4)\)). The variation of the growth rate profile with respect to \(p_0\) corresponding to the Fig. 13b is portrayed in Fig. 14. Also, the variation of the growth rate profile with respect to \(p_0\) in a generally nonlocal nonlinear media with various combinations of diffraction are shown in the supplementary material for \(\sigma =0.1\) in Fig. S8, \(\sigma =1.0\) in Fig. S9, \(\sigma =4.0\) in Fig. S10, and \(\sigma =10.0\) in Fig. S11.

Fig. 10
figure 10

Growth rate variation with wave number \(k_0\) and power \(p_0\) in weakly nonlocal nonlinear media for diffusion parameter \(\gamma =0.2\). a SD, b SD+TD, c SD+FD, and d SD+TD+FD

Fig. 11
figure 11

Growth rate variation with wave number \(k_0\) for power \(p_0=5.0\,\) in weakly nonlocal nonlinear media. Black color is for \(\gamma =0.0\), cyan color is for \(\gamma =0.2\), magenta color is for \(\gamma =0.4\), and blue color is for \(\gamma =0.6\). a SD, b SD+TD, c SD+FD, and d SD+TD+FD

Fig. 12
figure 12

Growth rate variation with wave number \(k_0\) and power \(p_0\) in weakly nonlocal nonlinear media for only SD case. The diffusion parameter a \(\gamma =0.0\), b \(\gamma =0.2\), c \(\gamma =0.4\) and d \(\gamma =0.6\). [corresponding to Fig. 11a ]

Fig. 13
figure 13

Growth rate variation with wave number \(k_0\) for power \(p_0=5.0\,\) in generally nonlocal nonlinear media. a only SD, b SD+TD, c SD+FD, and d SD+TD+FD. Black color is for \(\sigma =0.1\), cyan color is for \(\sigma =1.0\), magenta color is for \(\sigma =4.0\), and blue color is for \(\sigma =10.0\,\)

Fig. 14
figure 14

Growth rate variation with wave number \(k_0\) and power \(p_0\) in generally nonlocal nonlinear media for SD+TD. a \(\sigma =0.1\), b \(\sigma =1.0\), c \(\sigma =4.0\) and d \(\sigma =10.0\). [Corrosponding to Fig. 13b]

The general observations that TD creates asymmetry in the sideband and FD results in decay in intensity are somewhat similar to their temporal counterparts, i.e., third-order dispersion and fourth-order dispersion.

6 Conclusion

This paper showcases how higher order diffractions (SD, TD, and FD) impact the dynamics, stability analysis, and modulation instability of a narrow Gaussian optical beam in a nonlocal Kerr nonlinear medium. Apparently, TD has no effect on the critical energy of the soliton or corresponding potential profile, but FD reduces the required energy and widens the potential profile. If the initial energy of the beam is less than or larger than the critical energy of soliton formation, the beam width oscillates during propagation. The amplitude of the oscillations rises while the frequency falls due to the existence of FD; however, TD has no visible effect on those. The growth in the characteristic length of the response function reduces the amplitude of the oscillation and increases the frequency of the oscillation. Interestingly, TD is found to shift the beam center, while SD and FD have no such influence. Linear stability analysis of the governing equation of the system has been conducted to determine the parameter space for categorizing the equilibrium points. The modulation instability is investigated in the presence of HD. The presence of TD leads to asymmetric sidebands, while FD leads to the decay of the same.