1 Introduction

The classical standpoint of failure in soils considers a single limit failure surface in a stress space. For example, in the Mohr–Coulomb failure criterion, all the failure stress states lie on the failure surface. However, accumulative experimental evidence over the past few decades has indicated that this may not be true. This is because the other failure modes, namely strain localization and diffuse instability, may occur prior to the limit failure. The problem of shear localization is usually formulated as the possibility of the emergence of a weak discontinuity in the velocity field during quasi-static homogeneous deformation. Extensive research results have demonstrated that, for incrementally linear constitutive equation, the criterion for the occurrence of strain localization is consistent with the vanishing of the determinant of the acoustic tensor [9, 39, 50, 52]. This is particularly so for dense sands and over-consolidated clays. Meanwhile, diffuse instability arises without any accompanying inhomogeneous deformation and still requires an appropriate indicator of its occurrence. This is particularly so when various drainage conditions are considered [23, 27, 42, 45].

The onset of diffuse instability depends on factors including the initial density of soils, loading mode, and particularly, drainage conditions. Under undrained shear, static liquefaction or flow instability (deemed as a typical type of diffuse instability) generally occur in loose sand [3, 10]. However, in medium or dense sand, the quasi-stable (or limited flow) response, rather than diffuse instability, prevails. It is featured by a peak deviatoric stress followed by subsequent considerable dilation [20, 51, 54]. Influenced by the loading rate, permeability, and boundary conditions, the instability behavior of granular soil under a partially drained condition was observed to be different from that under fully undrained conditions [5, 19]. The effects of a drainage boundary condition on the potential instability of granular soil were investigated through experimental tests [29, 47, 48]. In particular, proportional strain loading (PSL) tests (wherein the ratio of the volumetric strain increment to the axial strain increment is maintained constant) are effective for examining the instability of sand under varying drainage conditions characterized by the imposed loading control parameter α. Mital and Andrade [31] demonstrated that flow liquefaction depends on both the imposed drainage condition and the density of the specimens. This indicates that loose soil that liquefies under undrained triaxial compression may be stable in contractive drainage condition. Moreover, dense soil that is stable under undrained triaxial compression may liquefy if expansive drainage is imposed. Substantial effort has also been invested to develop constitutive models, aiming at simulating the instability behavior of sand under axisymmetric PSL conditions. Based on a deviatoric hardening plasticity model, Lü et al. [27] indicated that the instability of sands depends on both the stress–dilatancy relation and the imposed strain proportion ratio. Using a state-dependent plasticity model, Lashkari and Yaghtin [22] simulated the instability of anisotropic samples under the proportional strain path. They observed that the stress ratio at the onset of diffuse instability is insensitive to Kc, the initial ratio of the horizontal to the vertical effective stress. However, no relevant microscopic data or convincing explanation was presented in their study.

Although a sand specimen remains stable before the deviatoric stress attains its peak value when sheared under the drained triaxial compression condition, such stability is conditional. The sand may lose its stability if the loading stress path differs [18]. A typical example is the constant shear drained (CSD) test, in which a continuous decrease in the mean normal stress is enforced. Such a test is aimed at mimicking the loading conditions of instability failures triggered in practical engineering, e.g., rainfall infiltration induced landslides and slope failure [14]. Under the CSD stress path, both experimental results [6, 49] and numerical simulations [36, 40] indicate that diffuse instability can occur in medium and dense sand. This is dramatically different from the failure scenario widely observed under undrained conditions. It has been demonstrated that Hill’s condition of stability [16] can be employed to predict the onset of instability in CSD tests [11, 46]. A few models have also been subsequently developed, aimed at simulating the instability observed in CSD tests [11, 42, 45]. Using the concept of loss of uniqueness, Alipour and Lashkari [1] predicted that two dissimilar scenarios and accordingly, two independent criteria for granular soils in loose and dense states may trigger instability under CSD, from the theoretical perspective. However, this prediction requires further scrutiny.

Although extensive data on the macroscopic instability behavior of sand has been obtained from both PSL tests and CSD tests, the micromechanical responses associated with the onset of instability and subsequent collapse of the microstructure are still unclear and yet to be explored. Moreover, although second-order work criterion has been proven to be general and powerful enough to detect diffuse failure occurrence [53], its application incorporated with a specific constitutive model of granular soils is still limited and deserves further investigations.

In the present study, both PSL and CSD tests are simulated using the discrete element method to explore the underlying mechanisms of diffuse instability under various drainage conditions. Although the macro-scale second-order work has been widely used as a legitimate indicator of the initiation of the instability, the counterpart in the micro-scale has not received comparable attention. A correlation between the burst of kinetic energy and onset of diffuse instability are established by considering the unbalanced forces of each particle. Therefore, based on both the macro- and micro-scale second-order work criterion, a variety of features accompanying the onset of diffuse instability of sand can be obtained. By introducing a fabric anisotropy variable, the correlation between fabric evolution and diffuse instability can be established. Based on the simulated DEM results, independent instability criteria are determined. Then, these criteria are incorporated with a newly developed anisotropic constitutive model with fabric evolution. A satisfactory agreement is obtained between the constitutive model’s predictions and the DEM simulations. The constitutive model also provides an accurate interpretation regarding why the diffuse instability occurs before the plastic limit. More importantly, the model enables the prediction of the onset of diffuse instability under various drainage conditions.

2 Numerical modeling methods and instability criteria

2.1 Discrete element simulations

The commercial particle flow code in three dimensions PFC3D [41] is employed to perform the DEM simulations. The key elements of the DEM are available in Cundall and Strack [8], and are not repeated herein. In this study, three-dimensional cubical specimens consisting of over 6700 particles are developed to serve as a representative volume element (RVE). Each specimen is enclosed by six massless, frictionless, and rigid walls.

Clumped particles are employed in this study. Each consists of two equal and partially overlapping basic spheres. The constant distance between the sphere centers is assigned to be 1.333r (r is the basic sphere radius). The clumped particle used is aimed at mimicking the non-spherical particles in PFC3D and therefore, at simulating a more realistic behavior of granular soils [28, 38, 56]. The equivalent particle radius is uniformly distributed within the range of 0.13 and 0.33 mm. A linear contact model with equal normal and tangential stiffness kn = ks =  105 N/m is used. The inter-particle friction coefficient μ is set as 0.5. The entire loading process is monitored to maintain the quasi-static condition. Meanwhile, the loading increment is applied when the ratio of the maximum unbalanced force to the average contact force is lower than the prescribed tolerance (set as 0.03% in the present study). To accelerate the simulation process and reduce the computational cost, numerical damping is employed to dissipate the kinetic energy of the particles. Note that this treatment does not affect the simulated responses as long as the quasi-static condition is fulfilled. In this study, a default value of the damping coefficient (ξ = 0.7) is employed. All the parameters used in the simulation are summarized in Table 1.

Table 1 Input parameters used in 3D numerical simulations

After all the particles have been generated randomly in terms of position and orientations within the domain bounded by the walls, the specimens are isotropically compacted to a low confining pressure of 10 kPa. The inter-particle friction μ is varied in the range from zero to 0.5 to adjust the initial density of the specimens. According to Yang et al. [58], the maximum void ratio can be estimated by generating samples with inter-particle friction coefficient μ = 0.5, and the minimum void ratio is obtained with μ = 0.0. Both these are under a reference state with confining pressure p′ = 10 kPa. Once the specimens have been generated, a common value of μ = 0.5 is restored, and all the specimens are isotropically consolidated (i.e., compressed) to desired confining pressures, e.g., 500 kPa. Various types of tests are conducted on the isotropically consolidated specimen, including conventional drained (CD) tests, constant volume (CV) tests, and PSL tests. Then, specimens with different initial void ratios are sheared to have the same stress ratio η = 0.5 under standard drained conditions, before they are sheared under CSD loading path. In addition, in order to investigate the influence of the stress anisotropy on the instability behavior, a few PSL tests are conducted on the anisotropically consolidated samples with the same mean normal stress p′ = 500 kPa and η = 0.5.

The stress tensor of the specimen enclosed by the rigid walls can be calculated by averaging the contact forces in the whole granular assembly [43] through the following equation:

$$\sigma_{ij} = \frac{1}{V}\sum\limits_{c = 1}^{k} {x_{i}^{c} F_{j}^{c} }$$
(1)

where V is the volume of the specimen in the current configuration, k is the total number of the contacts between particles and boundary walls, and Fc is the contact force exerted on the wall at contact point c with the coordinate denoted by xc.

In the triaxial condition, the mean normal stress p and deviatoric stress q are defined as follows:

$$\left\{ {\begin{array}{*{20}l} {p = \frac{1}{3}\left( {\sigma_{11} + \sigma_{22} + \sigma_{33} } \right)} \hfill \\ {q = \sqrt {{{\left[ {\left( {\sigma_{11} - \sigma_{22} } \right)^{2} + \left( {\sigma_{11} - \sigma_{33} } \right)^{2} + \left( {\sigma_{22} - \sigma_{33} } \right)^{2} } \right]} \mathord{\left/ {\vphantom {{\left[ {\left( {\sigma_{11} - \sigma_{22} } \right)^{2} + \left( {\sigma_{11} - \sigma_{33} } \right)^{2} + \left( {\sigma_{22} - \sigma_{33} } \right)^{2} } \right]} 2}} \right. \kern-0pt} 2}} } \hfill \\ \end{array} } \right.$$
(2)

respectively. Here, σ11, σ22, and σ33 are the major, intermediate, and major principal stresses, respectively.

The motions of the boundary walls can be applied to calculate the strains as follows:

$$\varepsilon_{z} = \frac{\Delta h}{{H_{0} }},\;\varepsilon_{x} = \frac{\Delta l}{{L_{0} }},\;\varepsilon_{y} = \frac{\Delta w}{{W_{0} }}$$
(3)

where H0, L0, and W0 are the height, length, and width, respectively, of the cubical specimen in the initial configuration. Their increments can be calculated by Δh = H0 − H, Δl = L0 − L, and Δw = W0 − W, respectively. Here, H, L, and W are the height, length, and width, respectively, of the specimen in the current configuration.

2.2 Second-order work criterion

To explore the underlying microscopic mechanism of the occurrence of diffuse instability, the evolution of both the macro-scale and micro-scale second-order work is presented and used to identify the onset of instability. A brief introduction to the method of calculation of the second-order work is provided below. Further discussions about the instability behavior of granular soil are presented in subsequent sections. It should be noted that the onset of instability in both the DEM simulations and constitutive modeling are analyzed in the framework of the second-order criterion.

2.2.1 Macro-scale second-order work

Hill [16] and Rudnicki and Rice [44] proposed the well-established criterion to identify the loss of stability in solids (termed Hill’s instability criterion). It has been corroborated by numerous experiments [5, 21, 29, 47, 48] and numerical simulations [4, 11, 33]. According to Hill’s condition of instability, the material is considered to be stable if the second-order work is strictly positive, i.e., d2W = dε > 0 for all feasible variations in the stress and strain. In a triaxial setting, the second-order work can be expressed as

$$d^{2} W = \text{d}q\text{d}\varepsilon_{q} + \text{d}p^{\prime}\text{d}\varepsilon_{v}$$
(4)

where the deviatoric stress q, effective mean stress p′, deviatoric strain εq, and volumetric strain εv are defined as follows: q  = σ1 − σ3, p′ = 1/3 (σ1′ + 2σ3′), εq = 2/3(ε1 − ε3), and εv = 1/3 (ε1 + 2ε3), respectively. In the present study, both the stress increments and strain increments can be registered every 5000 steps during the DEM simulations to calculate the macroscopic second-order work.

2.2.2 Microscopic second-order work

Nicot et al. [35] and Hadda et al. [15] calculated the second-order work in terms of the microscopic variables within the granular assembly by using the following equation:

$$d^{2} W^{m} = \underbrace {{\sum\limits_{c \in V} {\text{d}f_{i}^{c} } \text{d}l_{i}^{c} }}_{{d^{2} W_{c}^{{^{m} }} }} + \underbrace {{\sum\limits_{p \in V} {\text{d}f_{i}^{p} } \text{d}x_{i}^{p} }}_{{d^{2} W_{p}^{{^{m} }} }}$$
(5)

where fc is the contact force between particles; lc is the branch vector connecting the centers of the two contacting particles within the REV; and fp and xp are the resultant force and position of the particle p, respectively. It is noteworthy that both the creation and loss of the contacts during the calculation cycles can be accounted for by the term dfcdlc; herein, both the final contact force of the lost contact and the initial contact force of the new contact are considered as zero. In order to calculate the micro-scale second-order work, both the contact and particle information need to be stored at the beginning and end, respectively, of the calculation interval. In the present study, this interval is selected as 10,000 steps as a trade-off between the computational cost and accuracy.

Note that the first term of Eq. (5) (\({d}^{2} {{W}_{c}^{m}}\)) is related to the contacts, whereas the second term \({d}^{2} {{W}_{p}^{m}}\) is relevant to the particles and can be omitted if the entire process is quasi-static [15, 34]. However, when the quasi-static condition cannot be maintained and thus the particles move rapidly, an abrupt fluctuation of the term \({d}^{2} {{W}_{p}^{m}}\) is likely. This signifies the outburst of the kinetic energy within the REV. It should be emphasized that the diffuse instability failure mode corresponds to a homogeneous failure regime. Herein, no visible pattern of localization can be observed, and generally, a chaotic kinematic field dominates [34]. As the diffuse instability is essentially different from the strain localization failure, the evolution of the second term of Eq. (5) is presented separately to facilitate the identification of the initiation of the diffuse instability.

3 Simulation procedures

Among previous studies, the PSL test and CSD test are two typical examples that can simulate the continuous decrease in the mean principal stress p′ (and hence the occurrence of diffuse instability) under both partially drained and drained conditions. A brief introduction to these two typical loading schemes is presented below.

3.1 PSL path

Generally, fully drained and undrained conditions represent two extremes. They are considered valid only if the initial pore water pressure within the soil deposit is uniform throughout [47]. However, in the field conditions, the stress states and thus the loading path depends on the spatial variation in the excess pore pressure. Moreover, similar boundary constraints are likely to induce drainage conditions other than the above two ideal scenarios.

The imposed proportional strain paths of the tests can be described by the following conditions:

$$\left\{ {\begin{array}{*{20}l} {\text{d}\varepsilon_{1} > 0} \hfill \\ {\text{d}\varepsilon_{v} = \alpha \text{d}\varepsilon_{1} } \hfill \\ {\text{d}\varepsilon_{2} = \text{d}\varepsilon_{3} } \hfill \\ \end{array} } \right.$$
(6)

According to the sign convention in soil mechanics, the compression and volume contraction are both considered as positive, whereas the extension and volume dilation are negative. The volumetric response of the specimen (or the partial drainage) depends on the sign of the loading control parameter α as follows:

  1. 1.

    α = 0, undrained test (constant volume)

  2. 2.

    α < 0, dilation in volume

  3. 3.

    α > 0, contraction in volume

Note also that the stress path of α = 1 refers to the conventional oedometer test, in which the lateral strain is prohibited. Equation (6) prescribes a class of proportional strain path tests with constant strain ratio R = dε3/dε1 = (α− 1)/2. Fifteen PSL tests under various drainage conditions and different densities on both isotropically and anisotropically consolidated specimens are conducted, as listed in Table 2. The simulation results and more detailed discussions are presented in Sect. 4.

Table 2 Summary of proportional strain loading tests (PSL)

3.2 CSD tests

Failure is defined as the attainment of certain limit stress–strain states that cannot be sustained by the material. If these limit states are defined purely in a stress space, the classical definition of plastic limit criteria would be obtained. However, both experimental and theoretical evidence indicated that certain limit states obtained prior to the stress limit are mixed [1, 7, 37, 42]. The CSD test, with controlled deviatoric stress and volume variations, is probably the best-known example of such mixed limit states.

Four drained tests outlined in Table 3 are performed under constant deviatoric stress. The specimens with different initial void ratios are first isotropically consolidated to p0′ = 500 kPa. They are then sheared under conventional drained conditions to an identical stress ratio (η = 0.5), prior to the drained shear, where a constant deviatoric stress q is maintained until the onset of the instability, which is signified by the cross symbols in all the figures.

Table 3 Summary of constant shear drained tests (CSD)

4 DEM simulation results

4.1 Stress-deformation response

A series of proportional strain path tests were performed on specimens with varying α values and initial void ratios to simulate the instability behavior under partially drained condition. These are summarized in Table 2. As shown in Fig. 1, the presence of the peak for the deviatoric stress q is closely related to α. For the cases α ≤ 0, the smaller the value of the α is, the smaller the magnitude of the peak value of q. For the cases α > 0, q continuously increases without reaching a peak value. It should be noted that the onset of instability, signified by the symbol “X”, does not correspond to the presence of the peak of the deviatoric stress q. More detailed discussions about this asynchronism are given in Sect. 4.2. Special attention should be paid to the scenario of α < 0. Here, the volume expansion is imposed to simulate the process of water infiltration into the soil elements in the laboratory experiments [47, 48]. For the undrained case, the excess pore pressure is mainly induced by an increase in the shear stress. However, in the expansive drainage condition, the water inflow may contribute to the development of excess pore-water pressure in addition to the shear-induced pore pressure. Moreover, the effective stresses will hence be smaller than that of its undrained counterpart. This observation indicates that the undrained condition may not represent the most detrimental scenario in the field. This is evidenced by the fact that significantly lower shear strength is observed in the volumetric expansion (α < 0) under partial drainage condition.

Fig. 1
figure 1

Stress path of PSL tests a isotropically consolidated loose samples; b isotropically consolidated medium dense samples; c anisotropic loose samples η0 = 0.5

A comparison between Fig. 1a and b indicates that the initial void ratio also affects the instability of sand. For example, for the denser specimen (e0 = 0.674, α = 0), no peak of the deviatoric stress is observed, and a higher strength can be mobilized. In contrast, for the looser specimen with e0 = 0.706, a peak deviatoric stress is followed by a sharp decrease prior to the phase transformation state. Thereafter, the deviatoric stress is increased again toward the critical state. The state of the onset of instability is marked with a cross on these figures for each test, with the stress ratio ηins provided nearby. For a specified initial void ratio, it is observed that ηins is smaller when subjected to a more dilative loading path (smaller α). For example, for the medium dense samples, ηins= 0.79 (α = − 1) and ηins= 0.85 (α = − 0.5). Meanwhile, for a specified α value, ηins is always below the critical state stress ratio M (= 1.102) and is higher for denser specimens.

Recognizing that the initial stress state of the soils in situ is always anisotropic, a series of PSL tests are performed on specimens that are consolidated along the constant-p′ stress path up to η = 0.5 from the isotropic stress state, prior to the subsequent shearing. To eliminate the effect of the void ratio, the interparticle friction parameter is fine-tuned at the compaction stage. Thereby, it is ensured that both isotropically and anisotropically consolidated samples exhibit almost equal void ratio prior to the commencement of the PSL tests (Table 2). Figure 1c presents the effective stress paths of the anisotropic specimen, obtained from the PSL tests. It is observed that similar to the isotropic sample, the tendency of flow liquefaction increases as α varies from 0.5 to − 1. Moreover, full flow liquefaction and complete loss of shear strength occur for samples with α < 0. Moreover, a comparison between Fig. 1a and c indicates that the stress ratio at the onset of diffuse instability is almost insensitive to the stress anisotropy caused by the different consolidation paths. A more detailed discussion about this observation from the perspective of microstructure evolution is presented in Sect. 4.2.

Figure 2 presents the effective stress paths obtained from the CSD tests. The tests were conducted with varying initial void ratios and a fixed stress ratio η = 0.5. The stress ratio at the onset of the instability ηins is observed to be closely related to the initial void ratio. That is, the denser is the specimen, the larger is ηins on the initiation of the instability, and vice versa. For both the medium-dense and loose specimens (as shown in Table 3), ηins is always less than the critical stress ratio M. However, it could be higher than M for dense and very dense samples. Similar experimental results have been reported by Chu et al. [7] and Dong et al. [12]. Unlike the undrained tests presented in Fig. 1 (α = 0), by imposing the constant deviatoric stress drained path, all the specimens would undergo instability failure irrespective of their initial density. This indicates that the CSD stress path is more detrimental than the undrained path.

Fig. 2
figure 2

Stress path of CSD tests

Figure 3a presents the reduction in mean effective stress p′ against the axial strain. It is evident that the instability is accompanied by a sharp increase in the axial strain. This is in agreement with the experimental observations by Chu et al. [6, 7]. Figure 3b illustrates the evolution of the void ratio (or the volume change) with the mean normal stress. It is evident that all the specimens, including the loosest one, tend to dilate prior to the initiation of instability failure. This volumetric response is consistent with the experimental results reported by Dong et al. [12], Monkul et al., [30], and Daouadji et al., [13]. However, no dilation was observed by Chu et al. [7]. This inconsistency is speculated to be a result of the use of the “extremely loose” sample in their tests. Once the decrease in mean normal stress p′ is imposed, an “extremely loose” sample may immediately lose controllability of the deviatoric stress and hence the stability. Consequently, the second-order work transforms from positive to negative simultaneously owing to the volume contraction at the beginning of the CSD test. Moreover, the instability appears to be synchronized with the sharp variation in the volumetric response of sand; herein, the looser sand tends to contract, whereas the denser sand dilates. Based on this observation, two independent instability criteria can be determined for dense and loose sand, as presented in the following.

Fig. 3
figure 3

Evolutions of a axial strains and b void ratios in CSD tests

4.2 Instability analysis

The instability analysis is carried out within the framework of second-order work criterion. In terms of PSL tests, by combing Eq. (4) and (6) the second order work can be rewritten as:

$$\begin{aligned} {\kern 1pt} & \frac{1}{3}\left( {{\text{d}}\sigma_{1} + 2{\text{d}}\sigma_{3} } \right)\alpha {\text{d}}\varepsilon_{1} + \left( {{\text{d}}\sigma_{1} - {\text{d}}\sigma_{3} } \right)\left( {1 - \frac{\alpha }{3}} \right){\text{d}}\varepsilon_{1} \\ & \quad = {\kern 1pt} {\kern 1pt} \left( {{\text{d}}\sigma_{1} + {\text{d}}\sigma_{3} \left( {\alpha - 1} \right)} \right)\alpha {\text{d}}\varepsilon_{1} \\ & \quad = {\kern 1pt} \left( {{\text{d}}q + \alpha {\text{d}}\sigma_{3} } \right)\alpha {\text{d}}\varepsilon_{1} \\ \end{aligned}$$
(7)

It can be seen from Eq. (7) that the onset of instability corresponds to the vanishment of the term (dq + αdσ3) rather than the deviatoric stress q. Hence, to investigate the correspondence between the second-order work based instability criterion and the peak state of q or (q + ασ3) under the partially drained condition, the results of two example of anisotropic loose specimens with α = − 1 and = 0.5 are presented in Fig. 4a and b, respectively. It is observed that both the macroscopic and microscopic second-order work exhibits a similar trend with elapsed time. The second-order work becomes negative in synchrony with the peak of (q + ασ3), which occurs subsequent to the arrival of the peak value q. Such asynchronism between the peak value of q and the onset of instability implies that failure could occur even if the shear stress within the REV is lower than the shear strength (peak value of q). However, if there is no peak for (q + ασ3), as shown in Fig. 4b, the specimen would remain stable. The second-order work also remains positive during the entire loading process. Although no particle-scale information is obtained in the experiments, similar conclusions with respect to the macro-scale response and second-order work were obtained by Lancelot et al. [21] and Jrad et al. [29]. To reveal the underlying microscopic mechanism of the instability, the second term of the micro-scale second-order work d2W mp is also presented. It is observed that an abrupt disequilibrium (or fluctuation in the term d2W mp in Eq. (5)) occurs in synchrony with the vanishing of both the macro-scale and micro-scale second-order work. This signifies the onset of the instability. Note that the magnitude of the fluctuation in d2W mp is of the same order as the d2Wm and is almost zero when the specimen remains stable. Based on Eq. (5), the second term of the right hand side of the equation is related to inertial effects on the grain scale. This term, related to the imbalance forces, accounts for local inertial mechanisms that can be investigated from local kinetic energy. In DEM simulations, the evolutions of kinetic energy can be thoroughly examined. Total kinetic energy is the sum of the rotational and translational kinetic energies and can be calculated respectively as:

$$E_{kr} = \frac{1}{2}\sum\limits_{i = 1}^{{N_{p} }} {I_{i} } \omega_{i}^{2} ;\;E_{kt} = \frac{1}{2}\sum\limits_{i = 1}^{{N_{p} }} {m_{i} v_{i}^{2} } ;\;E_{k} = E_{kr} + E_{kt}$$
(8)

where Np is the total number of particles, mi, vi, and ωi are the mass, translational velocity, and rotational velocity of the particle i, respectively, and Ii is the moment of inertia of a clumped particle i. The evolutions of kinetic energy in the PSL tests are also plotted in Fig. 4. It can be seen that kinetic energy is negligibly small when the sample remains stable but an abrupt increase is observed at the occurrence of instability. This observation suggests that when the second-order work vanishes, a few infinitesimal perturbations will be sufficient to trigger the abrupt collapse of the granular system.

Fig. 4
figure 4

Evolutions of kinetic energy, macro- and micro-scale second-order work and their correspondence with q and q + ασ3 for anisotropic sepecimens η0 = 0.5 aα = − 1 bα = 0.5

To reflect the effect of the fabric anisotropy and its evolution on the instability, a fabric anisotropy variable A is employed. It was first introduced by Li and Dafalias [25] to quantify the combined influence of fabric and loading direction on soil behavior:

$$A = {\mathbf{F}}:{\mathbf{n}} = F{\mathbf{n}}_{F} :{\mathbf{n}} = FN$$
(9)

where n is the loading direction and F is a deviatoric contact-normal-based fabric tensor. F has two nontrivial invariants (i.e., norm F and direction nF of F), such that F can be expressed as follows [25, 56, 57]:

$${\mathbf{F}} = F{\mathbf{n}}_{F} ,\;F = \sqrt {{\mathbf{F}}:{\mathbf{F}}} ,\;{\mathbf{n}}_{F} :{\mathbf{n}}_{F} = 1,\;\text{tr}{\mathbf{n}}_{F} = 0$$
(10)

In the present study, the evolution of the state parameter A in the PSL tests on both isotropically- and anisotropically-consolidated specimens is illustrated in Fig. 5. In the figure, the initiation of instability is signified by the cross symbols. It is evident that for an equal loading control parameter α, A is almost the same at the occurrence of instability irrespective of the consolidation stress state. This is consistent with the observation in Fig. 1 that the stress ratio at the initiation of instability is almost insensitive to stress-induced anisotropy. This observation can be explained as follows: On one hand, it has been consistently evidenced by experimental observations [32, 55] and theoretical investigations [24, 25] that granular soil is more dilative when it is more anisotropic because the direction of the soil’s fabric is aligned with the loading direction. On the other hand, the plastic modulus of the anisotropic sample is smaller than that of the isotropic one owing to the higher stress ratio applied on the former [24, 27, 57]. When subjected to dilative volumetric strain, a smaller modulus promotes a decrease in the effective mean normal stress. This counteracts the increase owing to the dilatancy. These two competing effects renders almost identical stress ratio and fabric anisotropy variable at the onset of instability for both the isotropic and anisotropic samples with an equal α (Figs. 1 and 5). The following discussions are with regard to the samples exhibiting same initial stress ratio. At the early stage of the tests, the fabric anisotropy variable A is smaller for the test imposed with a larger α. This is because the PSL tests are purely strain-controlled, and the ratio between dε3 and dε1, i.e., R = dε3/dε1 = (α− 1)/2, is determined by α. As α increases, the strain increment dε3 gradually approaches dε1. This implies that the applied loading scheme is more isotropic and thus renders a smaller A at the early stages of the tests. However, as the loading process continues, the effective stress for more a contractive loading path (larger α) is more likely to increase. This defers the occurrence of instability and renders a larger A at the onset of instability.

Fig. 5
figure 5

Evolutions of fabric anisotropy variable A in both isotropic and anisotropic PSL tests

Similar to the partial drainage condition, the instability behavior of the sand under drained condition can be investigated in the framework of the second-order criterion. Considering the specimen with e0 = 0.667 as an example, the evolution of both the macro- and micro-scale second-order work is shown in Fig. 6. The occurrence of instability is also observed to be synchronized with the vanishing of both the second-order work, accompanied by an abrupt increase in the kinetic energy and the abrupt fluctuation in the second term of the micro-scale one. It is also observed that dq varies from zero to negative at the onset of instability. This is in accordance with the concept of “loss of controllability” [16, 17].

Fig. 6
figure 6

Evolutions of kinetic energy, macro- and micro-scale second-order work and their correspondence with q in CSD tests (e0 = 0.667)

Before the sample loses its stability in the CSD test, dq  = 0 such that the second-order work reduces to d2W  = dp′dεv. There are two possibilities for the initiation of instability with d2W  = dp′dεv = 0, i.e., dp′ = 0 or dεv = 0. To obtain the criteria triggering the vanishing of the second-order work and hence the instability of the specimen, the evolutions of the increments in the void ratio e and in the mean principal stress p′ are presented in Fig. 7a and b, respectively. Based on the known critical state line (CSL) (see Sect. 5), the state parameter defined by Been and Jefferies [2] ψ = e− ec can be calculated: ψ is positive for the loose sample and negative for the dense sample. That is, the initial state of the loose samples prior to the constant shear test is above the CSL, whereas those of the three dense samples are below the CSL. Figure 7 shows that for the samples below the CSL with ψ < 0, dεv< 0 holds, and dp′ becomes zero (no variation in p′) at the onset of instability. This signifies that d2W = 0. However, for the loose sample, the effective mean normal stress p′ continuously decreases (dp′ < 0 holds). Furthermore, it is the turning point of volume change from dilation to contraction (dεv= 0) at which d2W reduces from positive to zero. Hence, it can be concluded that the criterion triggering the onset of diffuse instability in CSD tests, signified by dq < 0, is state-dependent. Further discussions from the perspective of constitutive model are provided in Sect. 5.

Fig. 7
figure 7

Increments of a void ratio and b mean normal stress every 5000 steps in CSD tests

Figure 8 presents the evolutions of the fabric anisotropy variable in the CSD tests. It is observed that a looser sample has a larger A at the early stage of the CSD test. This is because a larger strain increment is imposed on a looser sample during the CD test prior to the CSD test to achieve a same desired stress state. As the mean normal stress continuously decreases, a denser sample is observed to lose stability at higher A. This indicates that a denser sample retains a larger stress ratio at the onset of instability and therefore is less prone to lose stability. Moreover, a larger A indicates a stronger tendency to dilate. This can explain the abrupt dilation of dense sand at the initiation of instability, as presented in Fig. 3b.

Fig. 8
figure 8

Evolutions of fabric anisotropy variable A in CSD tests

5 Insights from constitutive model predictions

As demonstrated in Sect. 4, the instability response of sand is observed to be closely related with the fabric anisotropy and its evolution. Hence, a constitutive model that can account for the fabric evolution is likely to be more suitable for simulating the instability behavior of sand. Therefore, a newly developed anisotropic model by Yang et al. [57] is employed to simulate the diffuse instability observed in the DEM tests. In this model, a novel expression of the fabric evolution rules is proposed. This allows for both the “hardening” and “softening” types of variation in the fabric norm, as observed in the DEM simulations. The engaged fabric anisotropy variable reflects the combined effect of the anisotropic consolidation and subsequent loading path and enables the simulation of the combined dilation–contraction deformation patterns.

This model is developed within the framework of the anisotropic critical state theory (ACST) developed by Li and Dafalias [25]. Further details are available in Yang et al. [57]. In this study, a simplified version is employed. It considers only the yielding caused by the increase in stress ratio η (q/p′) under a constant p′. The incremental relations of the model can be expressed by Eq. (11). The model parameters are listed in Table 4 and their detailed calibration procedure can be referred to Yang et al. [57].

$$\left\{ \begin{aligned} {\text{d}}q\ \\ {\text{d}}p' \\ \end{aligned} \right\} = \left[ {\left( {\begin{array}{*{20}c} {3G} & 0 \\ 0 & K \\ \end{array} } \right) - \frac{h\left( L \right)}{{K_{p} + 3G - K\eta D}}\left( {\begin{array}{*{20}c} {9G^{2} } & { - 3KG\eta } \\ {3KGD} & { - K^{2} \eta D} \\ \end{array} } \right)} \right]\left\{ \begin{aligned} {\text{d}}\varepsilon_{q} \\ {\text{d}}\varepsilon_{v} \\ \end{aligned} \right\}$$
(11)

In Eq. (11), G and K are the elastic shear and bulk moduli, respectively; Kp is the plastic modulus; η is the stress ratio; and D is the dilatancy of sand. The CD tests, PSL tests, and CSD tests are all simulated by the model using the single set of the model constants. The instability criteria for different drainage conditions and the corresponding simulation results are presented as follows:

Table 4 The model parameters

Because the constitutive model employed is developed within the framework of ACST, the critical state parameters of the granular soil need to be determined first. For non-cohesive soils such as sands and gravels, Li and Wang [26] recommended the following expression for CSL:

$$e_{cs} = e_{\varGamma } - \lambda \left( {\frac{p'}{{p_{a} }}} \right)^{\xi }$$
(12)

where eΓ, λ, and ξ are the material’s critical state constants and pa= 101 kPa is the atmospheric pressure. Based on the CD and undrained DEM simulations of the tests summarized in Table 5, a unique CSL fitting asymptotic states of nine drained and eight undrained tests can be obtained in both the qp′ and ep′ planes, as shown in Fig. 9. Therefore, the following parameters of CSL can be calibrated: eΓ = 0.767, λ = 0.0146, and M = 1.102. A default value ξ = 0.7 is assigned as it is not a sensitive parameter.

Table 5 Summary of both drained and undrained triaxial tests for the obtainment of critical state line in both p′q plane and ep′ plane
Fig. 9
figure 9

Critical state lines and the fitting curves in aq versus p′ plane; be versus p′ plane

Four CD test results conducted on specimens with different initial void ratios and a same confining pressure p0′ = 500 kPa are presented in Fig. 10. Other test results from the different initial conditions listed in Table 5 are not presented in the interest of brevity. Nonetheless, they are used to determine the CSL, as illustrated above. Based on the critical state parameters obtained, the tests simulated by the constitutive model employed include the CD tests, PSL tests, and CSD tests, as presented below. In the following comparison, DEM results are denoted by dotted symbols, and model responses are represented by solid lines.

Fig. 10
figure 10

Comparison of consititutive modeling and DEM results of conventional drained tests a evolution of stress ratio, η;b evolution of void ratio, e

5.1 Simulation of CD tests

For simplicity, the elasto-plastic relationship (Eq. (11)) can be re-expressed as

$$\left\{ {\begin{array}{*{20}c} {{\text{d}}q} \\ {{\text{d}}p'} \\ \end{array} } \right\} = \underbrace {{\left[ {\begin{array}{*{20}c} {\varPi_{qq} } & {\varPi_{qp} } \\ {\varPi_{pq} } & {\varPi_{pp} } \\ \end{array} } \right]}}_{\varPi }\left\{ {\begin{array}{*{20}c} {{\text{d}}\varepsilon_{q} } \\ {{\text{d}}\varepsilon_{v} } \\ \end{array} } \right\}$$
(13a)
$${\text{where}}\;\;\;\varPi = \left[ {\begin{array}{*{20}c} {\varPi_{qq} } & {\varPi_{qp} } \\ {\varPi_{pq} } & {\varPi_{pp} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {\frac{{3h\left( L \right)G\left( {K_{p} - K\eta D} \right)}}{{K_{p} + 3G - K\eta D}}} & {\frac{3h\left( L \right)KG\eta }{{K_{p} + 3G - K\eta D}}} \\ {\frac{ - 3h\left( L \right)KGD}{{K_{p} + 3G - K\eta D}}} & {\frac{{h\left( L \right)K\left( {K_{p} + 3G} \right)}}{{K_{p} + 3G - K\eta D}}} \\ \end{array} } \right]$$
(13b)

The test results simulated by the model are illustrated in Fig. 10. They match very well with the experimental results. It is observed that the model can simulate both the contractive and dilative response of sand, covering a wide range of the initial densities with satisfactory performance. Therefore, the model is used to simulate the PSL and CSD tests, as shown below.

5.2 Simulation of PSL tests

In this section, the mechanical behaviors of both anisotropic and isotropic samples subjected to PSL (α = − 1, − 0.5, 0, 0.25, and 0.5) are simulated. The developments of the shear stress q with the axial strain from these two series of tests are illustrated and compared with the DEM data in Fig. 11. It is observed that the model predictions match well with the DEM results for both isotropically and anisotropically consolidated samples with varying α values. It is noteworthy that Fin, a parameter employed in the present model and signifying the norm of the initial fabric, is set as 0.003 for the isotropically consolidated samples and as 0.122 for the anisotropically consolidated samples, based on the DEM simulation results, as shown in Table 4. As illustrated in Sect. 4.2, in PSL tests the second order work has been given in Eq. (7).

Fig. 11
figure 11

Comparison of consititutive modeling and DEM results of PSL tests a isotropic medium dense sample, e0 = 0.674; b anisotropic loose sample, e0= 0.704

Because the axial strain rate is non-zero, the only permissible solution is dq + αdσ3 = 0. This implies that the onset of diffuse instability predicted by the second-order work criterion coincides with the peak state of (q + ασ3). This has been verified by the above DEM simulations. However, the second-order work criterion is inconsistent with the plastic limit criterion. Typically, the plastic modulus Kp is positive at the onset of loading. Moreover, Kp = 0 temporally when the response of soils varies from strain hardening to softening, in synchrony with the peak state of q. However, in the PSL tests, such plastic limit criterion is valid only for the undrained tests. Hence, Kp= 0 (or the peak state of q) can no longer serve as a legitimate indicator of instability while considering a partially drained condition.

5.3 Simulation of CSD tests

CSD tests are conducted in the fully drained conditions. Herein, the deviatoric stress is maintained constant (dq = 0) until the sample loses its stability. As dq = 0 holds prior to the occurrence of diffuse instability, the criteria expressed by Eq. (4) reduces to

$$\text{d}p^{\prime} \cdot d\varepsilon_{v} = 0$$
(14)

There are two feasible solutions to Eq. (14): dp′  = 0 and v = 0. Based on the DEM results presented in Sect. 4.2, these two scenarios correspond to dense and loose sand, respectively.

For dense sand (dp′ = 0), the following equation is obtained:

$$\left\{ {\begin{array}{*{20}c} {{\text{d}}q} \\ {{\text{d}}p^{\prime}} \\ \end{array} } \right\} = \left\{ {\begin{array}{*{20}c} 0 \\ 0 \\ \end{array} } \right\} = \underbrace {{\left[ {\begin{array}{*{20}c} {\varPi_{qq} } & {\varPi_{qb} } \\ {\varPi_{pq} } & {\varPi_{pp} } \\ \end{array} } \right]}}_{\varPi }\left\{ {\begin{array}{*{20}c} {{\text{d}}\varepsilon_{q} } \\ {{\text{d}}\varepsilon_{v} } \\ \end{array} } \right\}$$
(15)

To obtain a non-trivial solution for Eq. (15) it is necessary that det(Π) = 0, i.e.,

$$\det \left( \varPi \right) = \varPi_{pp} \varPi_{qq} - \varPi_{pq} \varPi_{qp} = 0$$
(16)

where Πpp, Πqq, Πpq, and Πqp are uniquely associated with the specific constitutive model. Combining Eqs. (13a) and (16), the following equation is obtained:

$$\frac{{3KGh({\text{d}}L)}}{{K_{p} + 3G - K\eta D}}K_{p} = 0$$
(17)

Because the term 3KGh(dL) is always positive during the loading process, the onset of instability coincides with Kp = 0.

However, for loose sand (dεv= 0), Eq. (15) can be re-expressed as

$$\left\{ {\begin{array}{*{20}c} {{\text{d}}q} \\ {{\text{d}}p'} \\ \end{array} } \right\} = \left\{ {\begin{array}{*{20}c} 0 \\ {{\text{d}}p'} \\ \end{array} } \right\} = \underbrace {{\left[ {\begin{array}{*{20}c} {\varPi_{qq} } & {\varPi_{qb} } \\ {\varPi_{pq} } & {\varPi_{pp} } \\ \end{array} } \right]}}_{\varPi }\left\{ {\begin{array}{*{20}c} {{\text{d}}\varepsilon_{q} } \\ 0 \\ \end{array} } \right\}$$
(18)

By solving Eq. (18), the calculated volumetric strain rate becomes

$${\text{d}}\varepsilon_{v} = \frac{1}{\det \left( \varPi \right)}\varPi_{qq} {\text{d}}p' = 0\;\;\;{\text{if}}\;\det \left( \varPi \right) \, \ne \, 0$$
(19)

If dp′ = 0, det(Π) = 0. Therefore, the only feasible scenario is Πqq = 0. Based on Eq. (13b), Πqq = 0 is equivalent to Kp = KηD. Because the elastic volumetric strain increment \({\text{d}}\varepsilon _{v}^{e} = {\text{d}}p^{{\prime }} {\text{/}}K < 0\) owing to the continuous decrease in p′, the plastic volumetric strain increment \({\text{d}}\varepsilon _{v}^{p} = {\text{d}}\varepsilon _{v} - {\text{d}}\varepsilon _{v}^{e} = - {\text{d}}\varepsilon _{v}^{e}\) is positive. Hence, the dilatancy equation \(D = {\text{d}}\varepsilon _{v}^{p} {\text{/d}}\varepsilon _{q}^{p}\) and thus Kp is positive at the occurrence of instability in the present scenario. This clearly elucidates why diffuse instability can occur before plastic limit is attained.

Based on the above analyses, it can be concluded that a loose sample is subjected to contraction at the initiation of instability prior to the plastic limit. Meanwhile, the instability criterion of sands in dense state coincides with the plastic limit criterion. The comparisons between the DEM simulations and the constitutive model predictions are illustrated in Fig. 12. They exhibit satisfactory agreement.

Fig. 12
figure 12

Comparison of consititutive modeling and DEM results of CSD tests. a evolution of axial strain; b evolution of void ratio

6 Conclusions

The primary objective of the present study is to explore the underlying mechanisms triggering diffuse instability under various drainage conditions. Noting that PSL tests and CSD tests are two typical examples that replicate diffuse instability, both types of tests are conducted using the discrete element method. By introducing a fabric anisotropy variable A, the instability behavior of sand is observed to be closely related to its fabric evolution. A newly developed anisotropic plasticity model incorporating the fabric evolution is employed to simulate the numerical results from DEM. The following conclusions are drawn from this study:

  1. 1.

    Based on both macro- and micro-scale second-order work, the main features accompanying the onset of instability in both the PSL and CSD tests are obtained. In the PSL tests, special attention should be paid to the scenarios of α < 0. Here, the mobilized strength is lower than what is generally obtained from undrained tests (α = 0). In the CSD tests, all the specimens are observed to dilate prior to the onset of instability. Two independent criteria triggering the onset of diffuse instability in dense and loose sand are derived. For dense samples, it is dp′ = 0 that causes d2W to become zero. However, for loose samples, the abrupt contraction necessitates the transformation of d2W from positive to negative and hence the instability.

  2. 2.

    By introducing a fabric anisotropy variable A, the correlation between the instability behavior of sand and its fabric evolution can be established. In the PSL tests, for the samples having same initial stress ratio, the larger A is, the more dilative is the specimen and hence, lesser the tendency toward instability. However, both the stress ratio and the fabric anisotropy variable at the onset of instability are observed to be almost insensitive to stress-induced anisotropy. This can be explained by the counteraction between the enhanced dilation rendered by fabric anisotropy and the degraded plastic modulus owing to the initial stress ratio. In the CSD tests, the denser sample is observed to lose the stability at larger A in the later stage of the test. This indicates that a denser sample retains a larger stress ratio at the onset of instability and thus is less prone to instability. Moreover, a larger A indicates a strong tendency toward dilation. This may explain the abrupt dilation of dense samples at the initiation of instability.

  3. 3.

    Using the critical state parameters determined by the 17 drained and undrained tests (four of which liquefy), a newly developed anisotropic constitutive model incorporating the fabric evolution is employed to predict the mechanical behavior of the samples in both the PSL and CSD tests. The constitutive model can offer further insights into the failure criterion. In the PSL tests, the second-order work criterion is observed to be consistent with (dq + αdσ3) = 0 rather than dq = 0 (or Kp = 0). In the CSD tests, the second-order work criterion is identical to the plastic limit criterion for dense samples. Meanwhile, for loose samples, the instability failure predicted by the second-order work occurs when the plastic modulus is positive.