1 Introduction

Wellbore instability is a complex problem often encountered during drilling operations. Wellbore collapse, a type of wellbore instability, is caused by low mud pressure that cannot provide sufficient support to the rock in the wellbore wall. In the past, this problem has often been addressed by modelling studies.

The traditional model considers wellbore collapse as a problem of hole-edge stress concentration. Bradley (1979) presented the solution for the stress distribution around a borehole. The shear yield criterion is used to determine the minimum mud pressure (termed as critical mud pressure in this paper) that maintains a critical wellbore stability. The results showed that the distribution of in situ stresses and formation pressure are the major factors influencing critical mud pressure.

However, this traditional model does not consider the influence of porous flow on the stress distribution around the borehole. Detournay and Cheng (1988) established a model that considers the poroelastic response of a borehole in a nonhydrostatic stress field. Their analysis showed that the collapse is inside the rock, but not at the wellbore wall. To study the impact of elastic anisotropy on rock (e.g. anisotropy in the elastic modulus and Poisson’s ratio), Abousleiman and Cui (1998) established a similar poroelastic model for a transversely isotropic formation. Later, Ekbote and Abousleiman (2005) improved Abousleiman’s model by including coupled poro–chemo–thermal–mechanical effects. However, they assumed that the wellbore axis is perpendicular to the formation bedding plane. This limits the field in which these models can be applied.

Because, in general, the axis of a deviated wellbore is not necessarily vertical to the bedding plane, it is necessary to develop a method that overcomes this drawback. Aadnoy (1987) provided an analytical method for interpreting the stress distribution around a borehole in a transversely isotropic formation drilled at an arbitrary angle to the normal line of the bedding plane. Ong and Roegiers (1993) utilized this analytical method in conjunction with the consideration of rock strength anisotropy to assess how these anisotropies influence the shear yield around a wellbore. They believed that the Mohr–Coulomb yield criterion has limitations in analysing the critical mud pressure in a formation with weak planes because it is difficult to determine the stress distribution on the weak plane crossed by an arbitrarily oriented borehole. They used the generalized strength criterion proposed by Tsai and Wu (1971) and suggested that the shear yield was relatively unaffected by the elastic anisotropy. Worotnicki (1993) proved that the severe rock elastic anisotropy is highly distributed in shale formations by performing tri-axial experiments on 200 groups of core samples. Based on this, Li et al. (2011), Hou et al. (2013), and Lu et al. (2013a) analysed the influence of rock elastic anisotropy on the critical mud pressure for a formation with isotropic strength and demonstrated that this influence is significant.

The models described above did not study the effect of rock strength anisotropy on wellbore stability based on the Mohr–Coulomb yield criterion. Jin and Chen (1999a, b) proposed that the bedding plane is a weak plane, and for a specific situation, the rock on the wellbore will fail along the weak plane. Lee et al. (2012) used the coordinate system transformation method to obtain the stress distribution on the weak plane. He analysed the influence of the weak plane on the failure regions around a wellbore and noted that the wellbore shape may become rectangular if the mud pressure is too low. Lu et al. (2012, 2013b) studied the collapse phenomenon in the weak plane formation under porous flow and used a quantitative method to calculate the collapse intervals. However, the models described in those papers did not consider the rock elastic anisotropy and poroelastic anisotropy. In addition, due to the tectonic effect, multiple groups of natural fractures such as face and butt cleats are developed with different occurrences from the bedding plane, ignorance of which may produce erroneous results.

To summarize, a small amount of past researchers has focused on wellbore stability analyses for inclined boreholes that simultaneously considers the rock elastic anisotropy, rock strength anisotropy, poroelastic anisotropy, and multiple weak planes. Because transversely isotropic formations usually have several characteristics described (as above) at the same time, a model that includes all these effects is necessary. In this paper, two types of planes are considered. The bedding plane (BP) is considered to have transverse isotropy with a vertical axis of symmetry due to the sedimentation effect; it is also viewed as the first type of weak plane. The other type of weak plane is a natural fracture (NF, induced by the tectonic effect); its impact on the elastic behaviour of a rock is considered by using an equivalent continuum method. A coordinate system transformation is used to transform far-field stresses to a borehole coordinate system, and a complex variable method is used to derive the stress distribution around the borehole. The pore pressure is considered to be the formation pressure, and the method of Cheng (1997) is used to calculate the effective stress. His method provides a detailed description of the constitutive equations describing the incremental stress–strain and pressure–fluid content relationships for elastic anisotropic porous media. Six Biot coefficients have been defined that are not necessarily equal to each other, which are suitable for calculating effective stresses in elastic anisotropic porous media. The results of the effective stress distribution around a borehole will be projected to all types of weak planes, and the critical pressure is calculated for each type by using the Mohr–Coulomb yield criterion. Next, the maximum value among them will be identified as the final critical mud pressure for a specific wellbore trajectory. Finally, the impact of various factors on wellbore stability is analysed.

2 Stability Model for Transversely Isotropic Formation

The model considers three factors: 1. the poor cementation in BPs that leads to the rock yield along BPs; 2. the rock elastic anisotropy due to the sedimentation effect that modifies the stress distribution around a borehole compared with the isotropic model; and 3. the multiple weak planes that not only make the stress–strain relationship of a rock more complex, but also increase the possibility of rock failure along the weak planes, as shown in Fig. 1. Typically, a minimum mud pressure is to prevent collapse while a maximum one is to avoid exceeding the fracture gradient. Only the minimum mud pressure is discussed in this paper, and we define the pressure corresponding to the onset of shear yield as a critical mud pressure. The calculating procedure of the critical mud pressure can be divided into five steps.

Fig. 1
figure 1

A core with multiple weak planes

2.1 Step 1: Far-Field Stress Transformation

The Far-field stress distribution is transformed from the principal stress coordinate system (PCS) to the borehole coordinate system (BCS).

First, the far-field in situ stresses are transformed from PCS to a geographic coordinate system (GCS) as shown in Fig. 2a. GCS is defined as the positive side of X e -axis pointing to the north, Y e -axis to the east, and Z e -axis to the ground.

$${\varvec{\upsigma}}_{{\mathbf{e}}} = {\mathbf{R}}_{{\mathbf{1}}}^{{\mathbf{T}}} {\varvec{\upsigma}}_{{\mathbf{p}}} {\mathbf{R}}_{{\mathbf{1}}}$$
(1)
$${\mathbf{R}}_{{\mathbf{1}}} = \left\{ {\left. {\begin{array}{*{20}l} {\cos \alpha_{s} \cos \beta_{s} } \hfill & {\sin \alpha_{s} \cos \beta_{s} } \hfill & {\sin \beta_{s} } \hfill \\ { - \sin \alpha_{s} } \hfill & {\cos \alpha_{s} } \hfill & 0 \hfill \\ { - \cos \alpha_{s} \sin \beta_{s} } \hfill & { - \sin \alpha_{s} \sin \beta_{s} } \hfill & {\cos \beta_{s} } \hfill \\ \end{array} } \right\}} \right.;\quad {\varvec{\upsigma}}_{{\mathbf{p}}} = \left\{ {\left. {\begin{array}{*{20}c} {\sigma_{H} } & 0 & 0 \\ 0 & {\sigma_{h} } & 0 \\ 0 & 0 & {\sigma_{v} } \\ \end{array} } \right\}} \right.$$
(2)

where α s is the azimuth of the maximum horizontal principal stress and β s is the angle between the direction of the overburden stress and the Z e -axis; \({\varvec{\upsigma}}_{{\mathbf{p}}}\) is the far-field stress tensor under PCS; and \({\varvec{\upsigma}}_{{\mathbf{e}}}\) is the far-field stress tensor under GCS.

Fig. 2
figure 2

Coordinate system schematics: a PCS to GCS, b GCS to BCS

Second, the far-field stress distribution is transformed from GCS to the borehole coordinate system (BCS) as shown in Fig. 2b,

$${\varvec{\upsigma}}_{{\mathbf{b}}} = {\mathbf{R}}_{{\mathbf{2}}} {\varvec{\upsigma}}_{{\mathbf{e}}} {\mathbf{R}}_{{\mathbf{2}}}^{{\mathbf{T}}} = \left\{ {\begin{array}{*{20}l} {\sigma_{xx}^{b} } \hfill & {\tau_{xy}^{b} } \hfill & {\tau_{xz}^{b} } \hfill \\ {\tau_{yx}^{b} } \hfill & {\sigma_{yy}^{b} } \hfill & {\tau_{yz}^{b} } \hfill \\ {\tau_{zx}^{b} } \hfill & {\tau_{zy}^{b} } \hfill & {\sigma_{zz}^{b} } \hfill \\ \end{array} } \right\}$$
(3)
$${\mathbf{R}}_{{\mathbf{2}}} = \left\{ {\left. {\begin{array}{*{20}l} {\cos \alpha_{b} \cos \beta_{b} } \hfill & {\sin \alpha_{b} \cos \beta_{b} } \hfill & {\sin \beta_{b} } \hfill \\ { -\sin \alpha_{b} } \hfill & {\cos \alpha_{b} } \hfill & 0 \hfill \\ {-\cos \alpha_{b} \sin \beta_{b} } \hfill & {-\sin \alpha_{b} \sin \beta_{b} } \hfill & {\cos \beta_{b} } \hfill \\ \end{array} } \right\}} \right.$$
(4)

where \(\alpha_{\text{b}}\) and \(\beta_{\text{b}}\) are the azimuth and inclination angle of the wellbore and \({\varvec{\upsigma}}_{{\mathbf{b}}}\) is the far-field stress tensor under BCS. θ is the wellbore circumferential angle, which is measured from the positive side of X b counter clockwisely, as shown in Fig. 2b.

2.2 Step 2: Stress Distribution Around a Wellbore

The complex variable method is used to derive the stress distribution around a borehole under the far-field stress distribution \({\varvec{\upsigma}}_{{\mathbf{b}}}\). Lekhnitskii (1981) studied the stress distribution around a circular hole in an anisotropic plate under generalized plane strain assumption, which showed that this anisotropy can change the stress distribution to a considerable extent. The calculation method is extensively described by Amadei (1983) and Ong (1994) and we only give the outline of the solution for this problem.

2.2.1 Generalized Plane Strain Assumption

We consider an infinite transversely isotropic formation that is internally bounded by a cylindrical borehole of radius r. The far-field stresses around a borehole under BCS have been already derived in Step 1. For any arbitrarily directed borehole and principal stress distribution, all six far-field stress components may have nonzero values. In addition, in an anisotropic body, a plane of elastic symmetry parallel to the cross section of the borehole may not necessarily exist (i.e. the plane strain assumption fails). Instead, the cross sections will warp, but all identically. For these reasons, a generalized plane strain method is introduced. The idea that the cross sections will identically warp gives the essential of this method: all the gradients of stress and displacement along z direction are zero,

$${\text{Displacement:}}\,\frac{\partial u}{\partial z} = \frac{\partial v}{\partial z} = \frac{\partial w}{\partial z} = 0$$
(5)
$${\text{Stress:}}\,\frac{{\partial \tau_{xz} }}{\partial z} = \frac{{\partial \tau_{yz} }}{\partial z} = \frac{{\partial \sigma_{zz} }}{\partial z} = 0$$
(6)

According to the kinematic equations, the only strain that is zero is ɛ z . Equations (5) and (6) can then be used to simplify the expressions of the constitutive equations, equilibrium equations, and compatibility equations.

2.2.2 Compliance Tensor Conversion Between Different Coordinate Systems

We assume that \({\mathbf{A}}\) is the compliance tensor in the bedding plane coordinate system (BPCS, see Fig. 3), which is identical to the compliance tensor under BCS in the case when the borehole axis of a well is normal to the BP. It has a succinct form (see Eq. 8) and represents a linear stress–strain relationship of rocks (see Eq. 7), the components in which are the Hooke’s law constants.

Fig. 3
figure 3

Coordinate system schematics: bedding plane coordinate system (BPCS); stresses acting normal and tangential to the weak plane

$${\varvec{\varepsilon}} = {\mathbf{ A}}{\varvec{\sigma }}$$
(7)

In transversely isotropic formations, A can be expressed by the anisotropic elastic modulus E E , Poisson’s ratio ν ν and shear modulus G G . The symbols ∥, ⊥ mean the elastic parameters along and normal to the BP, respectively.

$${\mathbf{A}} = \left[ {\begin{array}{*{20}l} {\frac{1}{{E_{\parallel } }}} \hfill & { - \frac{{\nu_{\parallel } }}{{E_{\parallel } }}} \hfill & { - \frac{{\nu_{ \bot } }}{{E_{ \bot } }}} \hfill & 0 \hfill & 0 \hfill & 0 \hfill \\ { - \frac{{\nu_{\parallel } }}{{E_{\parallel } }}} \hfill & {\frac{1}{{E_{\parallel } }}} \hfill & { - \frac{{\nu_{ \bot } }}{{E_{ \bot } }}} \hfill & 0 \hfill & 0 \hfill & 0 \hfill \\ { - \frac{{\nu_{ \bot } }}{{E_{ \bot } }}} \hfill & { - \frac{{\nu_{ \bot } }}{{E_{ \bot } }}} \hfill & {\frac{1}{{E_{ \bot } }}} \hfill & 0 \hfill & 0 \hfill & 0 \hfill \\ 0 \hfill & 0 \hfill & 0 \hfill & {\frac{1}{{G_{ \bot } }}} \hfill & 0 \hfill & 0 \hfill \\ 0 \hfill & 0 \hfill & 0 \hfill & 0 \hfill & {\frac{1}{{G_{ \bot } }}} \hfill & 0 \hfill \\ 0 \hfill & 0 \hfill & 0 \hfill & 0 \hfill & 0 \hfill & {\frac{1}{{G_{\parallel } }}} \hfill \\ \end{array} } \right]$$
(8)

The shear modulus parallel to the plane G can be expressed by \(G_{\parallel } = \frac{{E_{\parallel } }}{{2(1 + \nu_{\parallel } )}}\). Through experiments, Batugin and Nirenburg (1972) noted that although G is theoretically an independent constant and is in no way related to the other elastic constants, it is possible to indicate an approximate formula linking G with the other parameters by \(G_{ \bot } = \frac{{E_{\parallel } E_{ \bot } }}{{E_{ \bot } (1 + 2v_{ \bot } ) + E_{\parallel } }}\). Thus, the total number of elastic constants reduces to four.

In contrast, when the axis of the borehole is not perpendicular to the BP, we can use the rotation technique to transform the compliance tensor from BPCS to BCS (Lekhnitskii 1981).

Suppose that \({\mathbf{x}}_{{\mathbf{b}}} {\mathbf{,y}}_{{\mathbf{b}}} {\mathbf{,z}}_{{\mathbf{b}}}\) are the vectors of the rectangular axes of BCS under GCS and \({\mathbf{x}}_{{{\mathbf{bp}}}} {\mathbf{,y}}_{{{\mathbf{bp}}}} {\mathbf{,z}}_{{{\mathbf{bp}}}}\), another set of vectors for BPCS whose direction cosines relative to \({\mathbf{x}}_{{\mathbf{b}}} {\mathbf{,y}}_{{\mathbf{b}}} {\mathbf{,z}}_{{\mathbf{b}}}\) are

$$\begin{aligned} l_{11} = \cos ({\mathbf{x}}_{{\mathbf{b}}} {\mathbf{,x}}_{{{\mathbf{bp}}}} ),l_{12} = \cos ({\mathbf{x}}_{{\mathbf{b}}} {\mathbf{,y}}_{{{\mathbf{bp}}}} ),l_{13} = \cos ({\mathbf{x}}_{{\mathbf{b}}} {\mathbf{,z}}_{{{\mathbf{bp}}}} ) \hfill \\ l_{21} = \cos ({\mathbf{y}}_{{\mathbf{b}}} {\mathbf{,x}}_{{{\mathbf{bp}}}} ),l_{22} = \cos ({\mathbf{y}}_{{\mathbf{b}}} {\mathbf{,y}}_{{{\mathbf{bp}}}} ),l_{23} = \cos ({\mathbf{y}}_{{\mathbf{b}}} {\mathbf{,z}}_{{{\mathbf{bp}}}} ) \hfill \\ l_{31} = \cos ({\mathbf{z}}_{{\mathbf{b}}} {\mathbf{,x}}_{{{\mathbf{bp}}}} ),l_{32} = \cos ({\mathbf{z}}_{{\mathbf{b}}} {\mathbf{,y}}_{{{\mathbf{bp}}}} ),l_{33} = \cos ({\mathbf{z}}_{{\mathbf{b}}} {\mathbf{,z}}_{{{\mathbf{bp}}}} ) \hfill \\ \end{aligned}$$
(9)

where the calculation method of \(\cos ({\mathbf{x}}_{{\mathbf{b}}} {\mathbf{,x}}_{{{\mathbf{bp}}}} )\), for example, is

$$\begin{aligned} {\mathbf{x}}_{{\mathbf{b}}} &= (\cos \alpha_{b} \cos \beta_{b} ,\sin \alpha_{b} \cos \beta_{b} , - \sin \beta_{b} ) \hfill \\ {\mathbf{x}}_{{{\mathbf{bp}}}} &= (\cos \alpha_{bp} \cos \beta_{bp} ,\sin \alpha_{bp} \cos \beta_{bp} , - \sin \beta_{bp} ) \hfill \\ \end{aligned}$$

\(\cos ({\mathbf{x}}_{{\mathbf{b}}} {\mathbf{,x}}_{{{\mathbf{bp}}}} ) = \frac{{{\mathbf{x}}_{{\mathbf{b}}} \cdot {\mathbf{x}}_{{{\mathbf{bp}}}} }}{{\left| {{\mathbf{x}}_{{\mathbf{b}}} } \right| \cdot \left| {{\mathbf{x}}_{{{\mathbf{bp}}}} } \right|}} = \cos \alpha_{b} \cos \beta_{b} \cos \alpha_{bp} \cos \beta_{bp} + \sin \alpha_{b} \cos \beta_{b} \sin \alpha_{bp} \cos \beta_{bp} + \sin \beta_{b} \sin \beta_{bp} ,\)

Under a rotation of the Cartesian coordinate system, the transformation tensor for the compliance tensor is given by Lekhnitskii (1981) as

$${\mathbf{Q}} = \left[ {\begin{array}{*{20}l} {l_{11}^{2} } \hfill & {l_{12}^{2} } \hfill & {l_{13}^{2} } \hfill & {l_{12} l_{13} } \hfill & {l_{13} l_{11} } \hfill & {l_{12} l_{11} } \hfill \\ {l_{21}^{2} } \hfill & {l_{22}^{2} } \hfill & {l_{23}^{2} } \hfill & {l_{23} l_{22} } \hfill & {l_{23} l_{21} } \hfill & {l_{22} l_{21} } \hfill \\ {l_{31}^{2} } \hfill & {l_{32}^{2} } \hfill & {l_{33}^{2} } \hfill & {l_{33} l_{32} } \hfill & {l_{33} l_{31} } \hfill & {l_{32} l_{31} } \hfill \\ {2l_{31} l_{21} } \hfill & {2l_{32} l_{22} } \hfill & {2l_{33} l_{23} } \hfill & {l_{33} l_{22} + l_{32} l_{23} } \hfill & {l_{33} l_{21} + l_{31} l_{23} } \hfill & {l_{31} l_{22} + l_{32} l_{21} } \hfill \\ {2l_{31} l_{11} } \hfill & {2l_{32} l_{12} } \hfill & {2l_{33} l_{13} } \hfill & {l_{33} l_{12} + l_{32} l_{13} } \hfill & {l_{33} l_{11} + l_{31} l_{13} } \hfill & {l_{31} l_{12} + l_{32} l_{11} } \hfill \\ {2l_{21} l_{11} } \hfill & {2l_{12} l_{22} } \hfill & {2l_{13} l_{23} } \hfill & {l_{13} l_{22} + l_{12} l_{23} } \hfill & {l_{13} l_{21} + l_{11} l_{23} } \hfill & {l_{11} l_{22} + l_{12} l_{21} } \hfill \\ \end{array} } \right]$$
(10)

If \({\mathbf{A}}^{{\prime }}\) is the compliance tensor in BCS and A in BPCS, following relation holds

$${\mathbf{A}}^{{\prime }} = {\mathbf{QAQ}}^{{\mathbf{T}}}.$$
(11)

2.2.3 Basic Equations

In BCS, the strains are linearly related to the stresses in the Cartesian coordinate system via the compliance tensor. The components of the compliance tensor \({\mathbf{A}}^{{\prime }}\) for a general anisotropic medium may not be zero after rotating the tensor A into BCS. Assuming that a ij is a component of the compliance tensor in BCS and ɛ zz  = 0, we can obtain the following constitutive equations,

$$\left[ {\begin{array}{*{20}c} {\varepsilon_{x} } \\ {\varepsilon_{y} } \\ 0 \\ {\gamma_{yz} } \\ {\gamma_{xz} } \\ {\gamma_{xy} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {a_{11} } & {a_{12} } & {a_{13} } & {a_{14} } & {a_{15} } & {a_{16} } \\ {a_{21} } & {a_{22} } & {a_{23} } & {a_{24} } & {a_{25} } & {a_{26} } \\ {a_{31} } & {a_{32} } & {a_{33} } & {a_{34} } & {a_{35} } & {a_{36} } \\ {a_{41} } & {a_{42} } & {a_{43} } & {a_{44} } & {a_{45} } & {a_{46} } \\ {a_{51} } & {a_{52} } & {a_{53} } & {a_{54} } & {a_{55} } & {a_{56} } \\ {a_{61} } & {a_{62} } & {a_{63} } & {a_{64} } & {a_{65} } & {a_{66} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\sigma_{x} } \\ {\sigma_{y} } \\ {\sigma_{z} } \\ {\tau_{yz} } \\ {\tau_{xz} } \\ {\tau_{xy} } \\ \end{array} } \right]$$
(12)

To solve 3D elastic problems, apart from the constitutive equations, three equilibrium equations and six compatibility equations are necessary. Using Eqs. (5) and (6), they can be simplified as follows

$$\begin{aligned} {\text{Equilibrium equations:}} & \frac{{\partial \sigma_{x} }}{\partial x} + \frac{{\partial \tau_{xy} }}{\partial y} = 0 \\ & \frac{{\partial \tau_{xy} }}{\partial x} + \frac{{\partial \sigma_{y} }}{\partial y} = 0 \\ & \frac{{\partial \tau_{xz} }}{\partial x} + \frac{{\partial \tau_{yz} }}{\partial y} = 0 \\ \end{aligned}$$
(13)
$$\begin{aligned} {\text{Compatibility equations:}} & \frac{{\partial^{2} \varepsilon_{xx} }}{{\partial y^{2} }} + \frac{{\partial^{2} \varepsilon_{yy} }}{{\partial x^{2} }} = \frac{{\partial^{2} \gamma_{xy} }}{\partial x\partial y} \\ & \frac{{\partial \gamma_{zx} }}{\partial y} - \frac{{\partial \gamma_{yz} }}{\partial x} = 0 \\ \end{aligned}$$
(14)

2.2.4 Stress Function and Beltrami–Michell Equations

To derive the exact formulae for the stresses around a borehole in anisotropic formations, two stress functions are defined as F(xy) and ψ(xy). These functions will be used to get the exact expressions for the stresses (Barber 2004), and they will make the stress field not only satisfy all the basic equations, but also fulfil the requirements of the boundary conditions (see Eqs. 23, 24). If these functions are related to the stress components in the following form, then Eq. (13) will be satisfied automatically,

$$\sigma_{x} = \frac{{\partial^{2} F}}{{\partial y^{2} }};\sigma_{y} = \frac{{\partial^{2} F}}{{\partial x^{2} }};\tau_{xy} = - \frac{{\partial^{2} F}}{\partial x\partial y};\tau_{xz} = \frac{\partial \psi }{\partial y};\tau_{yz} = - \frac{\partial \psi }{\partial x}$$
(15)

σ z can then be calculated from Eq. (12) after deriving all the other stress components. Substituting Eq. (15) into the constitutive relations, we obtain expressions for the strains in terms of the stress functions F(xy) and ψ(xy). By injecting them into Eq. (14), we obtain the generalized plane strain Beltrami–Michell equations of compatibility for an anisotropic body,

$$\begin{aligned} \left\{ {L_{4} L_{2} - L_{3}^{2} } \right\}F = 0 \hfill \\ \left\{ {L_{3}^{2} - L_{4} L_{2} } \right\}\psi = 0 \hfill \\ \end{aligned}$$
(16)

The differential operators L 2L 3L 4 are defined as follows

$$\begin{aligned} L_{2} & = \beta_{44} \frac{{\partial^{2} }}{{\partial x^{2} }} - 2\beta_{45} \frac{{\partial^{2} }}{\partial x\partial y} + \beta_{55} \frac{{\partial^{2} }}{{\partial y^{2} }} \\ L_{3} & = - \beta_{24} \frac{{\partial^{3} }}{{\partial x^{3} }} + (\beta_{25} + \beta_{46} )\frac{{\partial^{3} }}{{\partial x^{2} \partial y}} - (\beta_{14} + \beta_{56} )\frac{{\partial^{3} }}{{\partial x\partial y^{2} }} + \beta_{15} \frac{{\partial^{3} }}{{\partial y^{3} }} \\ L_{4} & = \beta_{22} \frac{{\partial^{4} }}{{\partial x^{4} }} - 2\beta_{26} \frac{{\partial^{4} }}{{\partial x^{3} \partial y}} + (2\beta_{12} + \beta_{66} )\frac{{\partial^{4} }}{{\partial x^{2} \partial y^{2} }} - 2\beta_{16} \frac{{\partial^{4} }}{{\partial x\partial y^{3} }} + \beta_{11} \frac{{\partial^{4} }}{{\partial y^{4} }} \\ \end{aligned}$$

where β ij is termed as the reduced strain coefficient by Lekhnitskii (1981),

$$\beta_{ij} = A_{ij}^{{\prime }} - \frac{{A_{i3}^{{\prime }}A_{j3}^{{\prime }} }}{{A_{33}^{{\prime }} }}\quad (i, j = 1,2,4,5,6)$$

2.2.5 Solutions for the Differential Equations

The differential equations can be solved using the method of characteristics, which can simplify the problems by converting higher-order differential equations to lower-order ones, and thus make the integration doable. This is achieved by replacing the stress functions F(xy) and ψ(xy) with the expression e x+χy in Eq. (16). After differentiation, they become the form,

$$f\left( \chi \right) = I_{4} \left( \chi \right)I_{2} \left( \chi \right) - I_{3}^{2} \left( \chi \right) = 0$$
(17)

with

$$\begin{aligned} I_{4} \left( \chi \right) & = \beta_{11} \chi^{4} - 2\beta_{16} \chi^{3} + \left( {2\beta_{12} + \beta_{66} } \right)\chi^{2} - 2\beta_{26} \chi + \beta_{22} \\ I_{3} \left( \chi \right) & = \beta_{15} \chi^{3} - \left( {\beta_{14} + \beta_{56} } \right)\chi^{2} + \left( {\beta_{25} + \beta_{46} } \right)\chi - \beta_{24} \\ I_{2} \left( \chi \right) & = \beta_{55} \chi^{2} - 2\beta_{45} \chi + \beta_{44} \\ \end{aligned}$$

Because Eq. (17) is a sixth-order polynomial equation, it has six roots that can be real or complex conjugates. Lekhnitskii (1981) noted that the roots of Eq. (17) are always complex or purely imaginary numbers, three of which are the conjugates of the other three. If we define the six roots as χ k k = 1, 2, 3, 4, 5, 6, then Eq. (16) can be decomposed into six first-order linear operators that can greatly reduce the complexity of the integration needed to solve the differential equations,

$$\begin{aligned} (D_{1} D_{2} D_{3} D_{4} D_{5} D_{6} )F(x,y) = 0 \hfill \\ (D_{1} D_{2} D_{3} D_{4} D_{5} D_{6} )\psi (x,y) = 0 \hfill \\ \end{aligned}$$
(18)

where \(D_{k} = \frac{\partial }{\partial y} - \chi_{k} \frac{\partial }{\partial x}\)The solutions of Eq. (18) have the following form (Ong 1994):

$$\begin{aligned} F(x,y) = \sum\limits_{k = 1}^{6} {F_{k} (z_{k} )} \hfill \\ \psi (x,y) = \sum\limits_{k = 1}^{6} {\psi_{k} (z_{k} )} \hfill \\ \end{aligned}$$
(19)

where F i (z i ) is an analytic function of the complex coordinates z i  = x i  + χ i y i . F i (z i ) is the spatial derivative of F i (z i ) with respect to z i .

Inserting Eq. (15) into Eq. (12) and then inserting Eq. (12) into Eq. (14), after lengthy algebra manipulation, a relationship between F(xy) and ψ(xy) can be obtained

$$L_{2} \psi_{k} (z_{k} ) + L_{3} F_{k} (z_{k} ) = 0$$
(20)

Based on Eqs. (19) and (20), Ong (1994) derived the general solution for the stress functions F(xy) and ψ(xy), as shown in Eq. (21). These two expressions satisfy all the basic equations. And the remaining work is to determine the unknown F i (z i ) and F i (z i ) according to the boundary conditions.

$$\begin{aligned} F(x,y) &= 2\text{Re} (F_{1} (z_{1} ) + F_{2} (z_{2} ) + F_{3} (z_{3} )) \hfill \\ \psi (x,y) &= 2\text{Re} (\lambda_{1} F_{1}^{{\prime }} (z_{1} ) + \lambda_{2} F_{2}^{{\prime }} (z_{2} ) + \frac{1}{{\lambda_{3} }}F_{3}^{{\prime }} (z_{3} )) \hfill \\ \end{aligned}$$
(21)

The complex variables λ i are expressed as follows

$$\lambda_{1} = - \frac{{I_{3} \left( {\chi_{1} } \right)}}{{I_{2} \left( {\chi_{1} } \right)}},\lambda_{2} = - \frac{{I_{3} \left( {\chi_{2} } \right)}}{{I_{2} \left( {\chi_{2} } \right)}},\lambda_{3} = - \frac{{I_{3} \left( {\chi_{3} } \right)}}{{I_{4} \left( {\chi_{3} } \right)}}$$
(22)

2.2.6 Stress Distribution Around the Wellbore

Inserting Eq. (21) into the stress function, Eq. (15), we deduce the form of borehole-induced stresses \(\sigma_{{ij,{\text{h}}}}\). The final stress distribution is the sum of the far-field stress tensor before drilling, the stress tensors induced by the removal of rocks, and by the mud pressure acting on the borehole wall (Amadei 1983; Ong 1994). This is achieved by first using the boundary conditions acting on the borehole (i.e. the far-field stress and mud pressure) to obtain the expression of \(\sigma_{{ij,{\text{h}}}}\), as shown in Eqs. (24) and (25), and then, superimpose the corresponding components of the far-field in situ stress tensor σ b ij on to \(\sigma_{{ij,{\text{h}}}}\) to deduce the complete stress field equations σ ij , as shown in Eq. (23).

$$\begin{aligned} \sigma_{x} & = \sigma_{xx}^{b} + \sigma_{{x,{\text{h}}}} = \sigma_{xx}^{b} + 2\text{Re} \left[ {\chi_{1}^{2} \varphi_{1}^{{\prime }} \left( {z_{1} } \right) + \chi_{2}^{2} \varphi_{2}^{{\prime }} \left( {z_{2} } \right) + \lambda_{3} \chi_{3}^{2} \varphi_{3}^{{\prime }} \left( {z_{3} } \right)} \right] \\ \sigma_{y} & = \sigma_{yy}^{b} + \sigma_{{y,{\text{h}}}} = \sigma_{yy}^{b} + 2\text{Re} \left[ {\varphi_{1}^{{\prime }} \left( {z_{1} } \right) + \varphi_{2}^{{\prime }} \left( {z_{2} } \right) + \lambda_{3} \varphi_{3}^{{\prime }} \left( {z_{3} } \right)} \right] \\ \tau_{xy} & = \tau_{xy}^{b} + \tau_{{xy,{\text{h}}}} = \tau_{xy}^{b} - 2\text{Re} \left[ {\chi_{1} \varphi_{1}^{{\prime }} \left( {z_{1} } \right) + \chi_{2} \varphi_{2}^{{\prime }} \left( {z_{2} } \right) + \lambda_{3} \chi_{3} \varphi_{3}^{{\prime }} \left( {z_{3} } \right)} \right] \\ \tau_{xz} & = \tau_{xz}^{b} + \tau_{{xz,{\text{h}}}} = \tau_{xz}^{b} + 2\text{Re} \left[ {\lambda_{1} \chi_{1} \varphi_{1}^{{\prime }} \left( {z_{1} } \right) + \lambda_{2} \chi_{2} \varphi_{2}^{{\prime }} \left( {z_{2} } \right) + \chi_{3} \varphi_{3}^{{\prime }} \left( {z_{3} } \right)} \right] \\ \tau_{yz} & = \tau_{yz}^{b} { + }\tau_{{yz,{\text{h}}}} { = }\tau_{yz}^{b} - 2\text{Re} \left[ {\lambda_{1} \varphi_{1}^{{\prime }} \left( {z_{1} } \right) + \lambda_{2} \varphi_{2}^{{\prime }} \left( {z_{2} } \right) + \varphi_{3}^{{\prime }} \left( {z_{3} } \right)} \right] \\ \sigma_{z} & = \sigma_{zz}^{b} + \sigma_{{z,{\text{h}}}} = \sigma_{zz}^{b} - \frac{1}{{A_{33}^{{\prime }} }}\left( {A_{31}^{{\prime }} \sigma_{{x,{\text{h}}}} + A_{32}^{{\prime }} \sigma_{{y,{\text{h}}}} + A_{34}^{{\prime }} \tau_{{yz,{\text{h}}}} + A_{35}^{{\prime }} \tau_{{xz,{\text{h}}}} + A_{36}^{{\prime }} \tau_{{xy,{\text{h}}}} } \right) \\ \end{aligned}$$
(23)

where φ 1(z 1) = F1(z 1), φ 2(z 2) = F2(z 2), \(\varphi_{3} (z_{3} ) = \frac{1}{{\lambda_{3} }}F_{3}^{{\prime }} (z_{3} )\), and σ b xx σ b yy σ b zz τ b xz τ b yz τ b xy are the far-field stresses under BCS.

The derivatives of the analytic functions φ i (z i ) can be solved by considering the boundary conditions on a wellbore, induced by removing the supporting rock and applying the mud pressure on the borehole wall. For this problem, the boundary conditions in Fourier series form (Ong and Roegiers 1993; Lu 1995) along the contour of the borehole are given by:

$$\begin{aligned} - \sigma_{x,h} \frac{dy}{ds} + \tau_{xy,h} \frac{dx}{ds} = \xi_{x} \hfill \\ - \tau_{xy,h} \frac{dy}{ds} + \sigma_{y,h} \frac{dx}{ds} = \xi_{y} \hfill \\ - \tau_{xz,h} \frac{dy}{ds} + \tau_{yz,h} \frac{dx}{ds} = \xi_{z} \hfill \\ \end{aligned}$$
(24)

where i is the complex number \(\sqrt { - 1}\), P w is the mud pressure, and

$$\begin{aligned} \xi_{x} = & \left( {\sigma_{xx}^{b} - P_{w} } \right)\cos \theta + \tau_{xy}^{b} \sin \theta - i\left( {\sigma_{xx}^{b} - P_{w} } \right)\sin \theta + i\tau_{xy}^{b} \cos \theta \\ \xi_{y} = & \left( {\sigma_{yy}^{b} - P_{w} } \right)\sin \theta + \tau_{xy}^{b} \cos \theta - i\left( {\sigma_{yy}^{b} - P_{w} } \right)\cos \theta + i\tau_{xy}^{b} \sin \theta \\ \xi_{z} = & \tau_{xz}^{b} \cos \theta + \tau_{yz}^{b} \sin \theta - i\tau_{xz}^{b} \sin \theta + i\tau_{yz}^{b} \cos \theta \\ \end{aligned}$$

Substituting \(\sigma_{{ij,{\text{h}}}}\) into Eq. (24) and integrating both sides of Eq. (24) with respect to the arc length give the expressions for the analytical functions φ i (z i ). We then expand the φ i (z i ) into series form, and by matching the unknown parameters in the series with the known boundary conditions Eq. (24), the exact expressions of the analytical functions and their derivatives can be determined.

$$\begin{aligned} \varphi_{1}^{{\prime }} \left( {{\text{z}}_{1} } \right) & = \frac{1}{{2G_{1} }} \times \left[ {D^{{\prime }} \left( {\lambda_{2} \lambda_{3} - 1} \right) + E '\left( {\chi_{2} - \lambda_{2} \lambda_{3} \chi_{3} } \right) + F '\lambda_{3} \left( {\chi_{3} - \chi_{2} } \right)} \right] \hfill \\ \varphi_{2}^{{\prime }} \left( {{\text{z}}_{2} } \right) & = \frac{1}{{2G_{2} }} \times \left[ {D^{{\prime }} \left( {1 - \lambda_{1} \lambda_{3} } \right) + E '\left( {\lambda_{1} \lambda_{3} \chi_{3} - \chi_{1} } \right) + F '\lambda_{3} \left( {\chi_{1} - \chi_{3} } \right)} \right] \hfill \\ \varphi_{3}^{{\prime }} \left( {{\text{z}}_{3} } \right) & = \frac{1}{{2G_{3} }} \times \left[ {D^{{\prime }} \left( {\lambda_{1} - \lambda_{2} } \right) + E '\left( {\chi_{1} \lambda_{2} - \chi_{2} \lambda_{1} } \right) + F '\left( {\chi_{2} - \chi_{1} } \right)} \right] \hfill \\ \end{aligned}$$
(25)

where

$$\begin{aligned} D^{{\prime }} = & \left( {P_{w} - \sigma_{xx}^{b} } \right)\cos \theta - \tau_{xy}^{b} \sin \theta - i\left( {P_{w} - \sigma_{xx}^{b} } \right)\sin \theta - i\tau_{xy}^{b} \cos \theta \\ E^{{\prime }} = & -\left( {P_{w} - \sigma_{yy}^{b} } \right)\sin \theta + \tau_{xy}^{b} \cos \theta - i\left( {P_{w} - \sigma_{yy}^{b} } \right)\cos \theta - i\tau_{xy}^{b} \sin \theta \\ F^{{\prime }} = & -\tau_{xz}^{b} \cos \theta - \tau_{yz}^{b} \sin \theta + i\tau_{xz}^{b} \sin \theta - i\tau_{yz}^{b} \cos \theta \\ G_{k} = & \left( {\chi_{k} \cos \theta - \sin \theta } \right)\left[ {\chi_{2} - \chi_{1} + \lambda_{2} \lambda_{3} \left( {\chi_{1} - \chi_{3} } \right) + \lambda_{1} \lambda_{3} \left( {\chi_{3} - \chi_{2} } \right)} \right] \\ \end{aligned}$$

This analytical solution has been validated by finite element method by Gaede et al. (2012). Results show that the borehole stresses computed from the numerical model and the analytical solution match almost perfectly for different borehole orientations and for several cases involving isotropic, transverse isotropic, and orthorhombic symmetries. In addition, they showed that though singularities in cases iii and iv (see Gaede’s paper) will be induced when β 11 = β 55, the equality can never be found in real world. In other words, this solution is valid with no restriction on borehole orientations or rock elastic properties when doing wellbore stability modelling (in real world).

2.3 Step 3: Solution to the Critical Mud Pressure

First, the stress distribution around a borehole will be transformed from BCS to GCS and then from GCS to BPCS (see Fig. 3).

$${\varvec{\upsigma}}_{{{\mathbf{bp}}}} = {\mathbf{R}}_{{\mathbf{3}}} {\mathbf{R}}_{{\mathbf{2}}}^{{\mathbf{T}}} {\varvec{\upsigma}}_{{\mathbf{b}}}^{{{\mathbf{eff}}}} {\mathbf{R}}_{{\mathbf{2}}} {\mathbf{R}}_{{\mathbf{3}}}^{{\mathbf{T}}} = \left\{ {\begin{array}{*{20}c} {\sigma_{xx}^{bp} } & {\tau_{xy}^{bp} } & {\tau_{xz}^{bp} } \\ {\tau_{yx}^{bp} } & {\sigma_{yy}^{bp} } & {\tau_{yz}^{bp} } \\ {\tau_{zx}^{bp} } & {\tau_{zy}^{bp} } & {\sigma_{zz}^{bp} } \\ \end{array} } \right\}$$
(26)
$${\mathbf{R}}_{{\mathbf{3}}} = \left\{ {\left. {\begin{array}{*{20}l} {\cos \alpha_{bp} \cos \beta_{bp} } \hfill & {\sin \alpha_{bp} \cos \beta_{bp} } \hfill & {\sin \beta_{bp} } \hfill \\ { - \sin \alpha_{bp} } \hfill & {\cos \alpha_{bp} } \hfill & 0 \hfill \\ { - \cos \alpha_{bp} \sin \beta_{bp} } \hfill & { - \sin \alpha_{bp} \sin \beta_{bp} } \hfill & {\cos \beta_{bp} } \hfill \\ \end{array} } \right\}} \right.$$
(27)
$${\varvec{\upsigma}}_{{\mathbf{b}}}^{{{\mathbf{eff}}}} = \left\{ {\begin{array}{*{20}l} {\sigma_{x}^{eff} } \hfill & {\tau_{xy}^{eff} } \hfill & {\tau_{xz}^{eff} } \hfill \\ {\tau_{xy}^{eff} } \hfill & {\sigma_{y}^{eff} } \hfill & {\tau_{yz}^{eff} } \hfill \\ {\tau_{xz}^{eff} } \hfill & {\tau_{yz}^{eff} } \hfill & {\sigma_{z}^{eff} } \hfill \\ \end{array} } \right\} = \left\{ {\begin{array}{*{20}l} {\sigma_{x} } \hfill & {\tau_{xy} } \hfill & {\tau_{xz} } \hfill \\ {\tau_{xy} } \hfill & {\sigma_{y} } \hfill & {\tau_{yz} } \hfill \\ {\tau_{xz} } \hfill & {\tau_{yz} } \hfill & {\sigma_{z} } \hfill \\ \end{array} } \right\} - \alpha p$$
(28)

where \(\alpha_{\text{bp}} + \pi ,\beta_{bp}\) correspond to the dip direction and dip angles of the BP; p is the pore pressure; α is the Biot coefficient for an isotropic formation; and \({\varvec{\upsigma}}_{{{\mathbf{bp}}}}\) is the stress tensor under BPCS.

Two Mohr–Coulomb yield criteria are used to calculate the critical mud pressure that corresponds to the intact rock and weak plane, respectively. Using these criteria do not mean that the rock will collapse from the wellbore. Instead, it marks the onset of compressive shear yield, which is termed as the critical mud pressure.

$${\text{Intactrock:}}\,\sigma_{1}^{\hbox{max} } = \sigma_{3}^{\hbox{min} } + 2(S_{o} + u_{o} \sigma_{3}^{\hbox{min} } )(\sqrt {(1 + u_{o}^{2} )} + u_{o} )$$
(29)
$${\text{Weak plane:}}\,\sqrt {\left( {\tau_{zx}^{bp} } \right)^{2} + \left( {\tau_{zy}^{bp} } \right)^{2} } = S_{bp} + u_{bp} \sigma_{zz}^{bp}$$
(30)

where \(S_{\text{o}} ,u_{\text{o}}\) are the cohesive strength and coefficient of friction of the intact rock, while S bpu bp are the cohesive strength and coefficient of friction on the BP, respectively.

Following the method proposed by Zhang et al. (2015), the maximum and minimum principal stresses (σ max1 and σ min3 ) on the borehole are mathematically equivalent to the maximum and minimum eigenvalues of the stress tensor Eq. (28).

Figure 4 shows the detailed calculation algorithm for critical mud pressure. The main improvements done for its predecessor embody two aspects: 1. the introduction of the elastic anisotropy and 2. a correction made for the failure criterion Eq. (30). This correction is necessary when the in situ stresses are highly inhomogeneous or the strength on weak planes is too low (see Sect. 3.5 for discussion). Then, for a specific trajectory, the selected mud pressure can be expressed as:

$$P_{c} = \hbox{max} (P_{co} ,P_{cw} )$$
(31)

where P co P cw represent the mud pressure derived by satisfying each (intact rock and weak plane) yield criterion, respectively.

Fig. 4
figure 4

Calculation algorithm for critical mud pressure, modified after Lee et al. (2012)

2.4 Step 4: Considering Multiple Weak Planes

Due to the tectonic effects such as tectonic compression and extension, NFs are often developed in a formation. Their occurrence is not identical to the BP, which means that NFs must be viewed as another type of weak plane. Ignoring the presence of them is not an option because, first, their role is dominant in the stress–strain relationship of the rock, and second, the poor cementation inside them may cause rock yield along a NF. Here, we first consider the influence of NFs on the elastic behaviour of the rock through an equivalent method and then derive the solution for the critical mud pressure by using multiple weak plane yield criterion. Figure 5 shows a group of NFs in BPCS.

Fig. 5
figure 5

Natural fracture coordinate system (NFCS) in BPCS

2.4.1 The Influence of NFs on the Compliance Tensor

Rocks with NFs can be viewed as jointed rocks that contain one or several sets of discontinuities. These discontinuities create anisotropy in rocks’ response to loading and unloading (Amadei 1983). One convenient way to model this response is to treat the rock as an equivalent anisotropic continuum (Wu 1988): the deformations of the equivalent continuum include the deformation of the intact rock as well as the deformation resulting from the displacements on discontinuities. The application of this model requires the estimation of a so-called representative elementary volume (REV). In the REV, the rock mass can be considered to be statistically homogeneous in the sense that an increase in this volume does not change the mean stresses and displacements. This condition can be well fulfilled if the spacing between the discontinuities is smaller than 1/8–1/10 of the characteristic dimension of the considered structure (Wittke 2014). Therefore, this theory is most suitable for borehole stability analysis in a coal seam, where face and butt cleats prevail. The space between and the size of the cleats are often of the magnitude from um to mm (Su et al. 2001; Acosta et al. 2007), which is considerably smaller than the radius of borehole, and thus satisfy the ‘1/8–1/10’ requirement.

Moreover, when the spacing between discontinuities has the same dimension with the considered structure, the equivalent continuum theory is still applicable. For example, Chiu et al. (2012) and Xu et al. (2015) made comparisons between the equivalent continuum theory and discrete model theory via numerical simulations. In their studies, the two methods are used to simulate rock core’s and tunnel’s deformation under complex confining stress state. Moderate distributed fractures are inserted into the core and tunnel, and the space between fractures is in the same dimension with the structure. It shows that in both the near discontinuity and far from discontinuity regions, stress and displacement distribution predicted by these two models are close to each other. As a result, for borehole stability analysis, this approach can still be used if the average spacing between NFs is around or less than 10 cm, which is in the same scale with the borehole diameter. However, master joints and faults with large spacing must be modelled discretely (Wittke 2014).

In this approach, the parameters of the fractures, such as the average spacing, normal and shear stiffness as well as dilation angle, are used to study their additional influence on the constitutive behaviour of the rocks.

Goodman (1976) and Amadei (1983) systematically studied the influence of joints on the elastic behaviour of rocks, by an equivalent anisotropic medium concept. Experimental result for the behaviour of joints under changing normal stress with constant shear stress gives the hyperbola-shaped deformation curve (Fig. 6a) while an idealized shear deformation model for the behaviour under changing shear stress with constant normal stress is shown in Fig. 6b. Correspondingly, a secant normal stiffness k n and a unit shear stiffness k s are proposed to resemble the deformation behaviour of joints in the elastic region. In addition, Zhu and Wang (1992) believed that the dilation may originate from surface roughness, which connotes a thickening of a joint. He used dilation angle ω to represent the normal displacement induced by a joint shear deformation, and it is expressed by \(\frac{\tau }{{k_{s} }}\tan (\omega )\). Thereafter, the two stiffness parameters k s and k n , dilation angle ω, and average normal spacing s between fractures can be used to calculate the compliance incremental tensor in NFCS, as shown in Eq. (32).

$${\mathbf{A}}_{{{\mathbf{nf}}}} = \left[ {\begin{array}{*{20}c} 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & {\frac{1}{{k_{n} s}}} & {\frac{ - 1}{{k_{s} s}}\tan (\omega )} & {\frac{ - 1}{{k_{s} s}}\tan (\omega )} & 0 \\ 0 & 0 & {\frac{ - 1}{{k_{s} s}}\tan (\omega )} & {\frac{1}{{k_{s} s}}} & 0 & 0 \\ 0 & 0 & {\frac{ - 1}{{k_{s} s}}\tan (\omega )} & 0 & {\frac{1}{{k_{s} s}}} & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ \end{array} } \right]$$
(32)

where \(\frac{1}{{k_{n} s}}\) represents the induced strain along the z-axis by a unit z-axis normal stress; \(\frac{1}{{k_{s} s}}\) represents the induced strain along the x (or y)-axis by a unit x (or y)-axis shear stress; \(\frac{ - 1}{{k_{s} s}}\tan (\omega )\) denotes the induced strain along z-axis by a unit x (or y)-axis shear stress.

Fig. 6
figure 6

Normal and shear stiffness test of a single NF: a normal stiffness, b shear stiffness [reproduced after Amadei (1983)]

Using the transformation method illustrated previously, we can transform the increment \({\mathbf{A}}_{{{\mathbf{nf}}}}\) under NFCS to \({\mathbf{A}}'_{{{\mathbf{nf}}}}\) under BPCS.

Therefore, the overall compliance tensor in a transversely isotropic formation considering NFs under BPCS could be expressed as follows:

$${\varvec{\upvarepsilon}}_{{{\mathbf{total}}}} {\mathbf{ = \varepsilon }}_{{{\mathbf{formation}}}} {\mathbf{ + \varepsilon }}_{{{\mathbf{nf}}}} {\mathbf{ = A\sigma + A'}}_{{{\mathbf{nf}}}} {\mathbf{\sigma = A}}_{{{\mathbf{total}}}} {\varvec{\upsigma}}$$
(33)

We then replace the tensor \({\mathbf{A}}\) with \({\mathbf{A}}_{{{\mathbf{total}}}}\) in Eq. (11), so that the constitutive relations considering the influence of NFs on the elastic response of the rock can be derived.

2.4.2 Critical Mud Pressure Considering Multiple Weak Planes

Because we consider NFs as another type of weak plane, Eq. (31) should be changed into the form,

$$P_{c} = \hbox{max} (P_{co} ,P_{cw1} ,P_{cw2} )$$
(34)

where P cw1P cw2 are the critical mud pressure calculated according to the yield criteria of the BP and NF, respectively. If there are other groups of NFs with different occurrences, we can use the same method outlined in Step 4 to include them.

2.5 Step 5: Considering Poroelastic Effect

Because the formation is a porous medium, the effective stress should be used in calculations. Cheng (1997) studied the effective stress distribution in an anisotropic formation and provided the expression of the Biot coefficients as follows,

$$\alpha_{ij} = \delta_{ij} - \frac{{M_{ijkk} }}{{3K_{s} }}$$
(35)

where M ijkk are the drained coefficients in the stiffness tensor, K s is the grain modulus, and α ij are the Biot coefficients.

For engineering convenience, we transform the standard tensor above to the below form using Voigt notation,

$$\begin{aligned} \left[ {\begin{array}{*{20}c} {\sigma_{x} } \\ {\sigma_{y} } \\ {\sigma_{z} } \\ {\tau_{yz} } \\ {\tau_{xz} } \\ {\tau_{xy} } \\ \end{array} } \right] = {\mathbf{M}}\left[ {\begin{array}{*{20}c} {\varepsilon_{x} } \\ {\varepsilon_{y} } \\ {\varepsilon_{z} } \\ {\gamma_{yz} } \\ {\gamma_{xz} } \\ {\gamma_{xy} } \\ \end{array} } \right] + p\left[ {\begin{array}{*{20}c} {\alpha_{1} } \\ {\alpha_{2} } \\ {\alpha_{3} } \\ {\alpha_{4} } \\ {\alpha_{5} } \\ {\alpha_{6} } \\ \end{array} } \right]\begin{array}{*{20}c} {\alpha_{i} = 1 - \frac{{\sum\nolimits_{j = 1}^{3} {M_{ij} } }}{{3K_{s} }},\begin{array}{*{20}c} {if} & {i = 1,2,3} \\ \end{array} } \\ {\alpha_{i} = - \frac{{\sum\nolimits_{j = 1}^{3} {M_{ij} } }}{{3K_{s} }},\begin{array}{*{20}c} {if} & {i = 4,5,6} \\ \end{array} } \\ \end{array} \hfill \\ \hfill \\ \hfill \\ \end{aligned}$$
(36)

where \({\mathbf{M = }}({\mathbf{A}}_{{{\mathbf{total}}}} )^{ - 1}\), and \({\mathbf{A}}_{{{\mathbf{total}}}}\) is defined in Eq. (33).

Therefore, to include the poroelastic anisotropy effect, the effective stress tensor around the wellbore in Eq. (28) can be modified to the form,

$${\varvec{\upsigma}}_{{\mathbf{c}}}^{{{\mathbf{eff}}}} = \left\{ {\begin{array}{*{20}c} {\sigma_{x} } & {\tau_{xy} } & {\tau_{xz} } \\ {\tau_{xy} } & {\sigma_{y} } & {\tau_{yz} } \\ {\tau_{xz} } & {\tau_{yz} } & {\sigma_{z} } \\ \end{array} } \right\} - \left\{ {\begin{array}{*{20}c} {\alpha_{1} } & {\alpha_{6} } & {\alpha_{5} } \\ {\alpha_{6} } & {\alpha_{2} } & {\alpha_{4} } \\ {\alpha_{5} } & {\alpha_{4} } & {\alpha_{3} } \\ \end{array} } \right\}p$$
(37)

Contrary to the isotropic case, it shows that the Biot coefficients α 4α 5α 6 may not necessarily be zero and α 1α 2α 3 may not necessarily equal to each other. This means that pore pressure not only has an effect on principal stresses, but could also influence shear stresses. Due to the elastic anisotropy of the rock, many shear effects are induced by the normal stress because of the shear–tension coupling effect. This is why static fluid pressure can transmit shear stresses.

3 Critical Mud Pressure Analysis and Discussion

Using the developed simulator, a parametric study was carried out based on the inputs from Table 1, in which, for validation purposes, some of them come from the published work by Lee et al. (2012). The parameters studied include controllable parameters, such as mud pressure and well trajectory (wellbore azimuth and inclination angle), and uncontrollable effects, such as weak plane occurrence, formation intrinsic elastic anisotropy, and stiffness of NFs. We assume the formation is transversely isotropic due to the sedimentation effect and made of well-developed BPs (the first type of weak plane). In addition, a group of NFs (the second type of weak plane) is developed due to the tectonic effect.

Table 1 Mechanical parameters of rock mass and far-field in situ stresses

3.1 The Influence of Rock Elastic Anisotropy

We define two anisotropy indexes N = E /E and M = ν /ν to represent the degree of elastic anisotropy of the rock. In general, an elastic property measured normal to the BP is weaker than ones parallel to the BP, which means that N ≤ 1 and M ≥ 1. As a result, only this situation is defined as a BP-induced elastic anisotropy. Other situations such as N > 1 or M < 1 are not discussed here.

In Fig. 7a, we show the case of the isotropic formation under a stress regime of strike-slip fault. The polar plot shows that the relative stable trajectory is a function of inclination and azimuth angles. The concentric circles represent increasing inclination, with the outer boundary representing horizontal wells. The numbers around the outer circle correspond to the borehole azimuth, which is measured clockwisely from the north. The central area in the plot, which corresponds to a vertical well, is observed to be relatively unstable, while the most stable drilling direction is parallel to the maximum horizontal principal stress with a high inclination. The plot is symmetrical, and the results are in accordance with determined by the elastic isotropic models (Last 1996; Lee et al. 2012).

Fig. 7
figure 7

Polar plots of critical mud pressure: a isotropic case, b BP-induced elastic anisotropy (N = 0.5, M = 1)

In Fig. 7b, to study the BP-induced anisotropy, the elastic modulus normal to the isotropic plane is reduced to 0.5 times that of the in-plane one (N = 0.5). Compared with Fig. 7a, the high-value area shifts from the plot centre to the northeast and southwest, the variation of which is about 1 MPa. The pressure values on the other grids, however, maintain almost constant. The same trend is observed when set M = 2, but N = 1, showing that the anisotropy induced by elastic modulus or Poisson’s ratio alone with the given degree (N = 0.5 or M = 2) cannot have a significant impact on the magnitude in the polar plot. We then combined both the indexes.

The parameters used in Fig. 8a are the same as those chosen for Fig. 7b, except that the Poisson’s ratio normal to the isotropic plane is set to be two times that of the in-plane one. Compared with Fig. 7b, they look very alike except that the magnitude of pressure decreases almost uniformly at every grid in Fig. 8a, indicating that the elastic anisotropy induced by the sedimentation effect does help keep a wellbore stable for any well trajectory. Care should be taken, however, that the selection of the parameters in Table 1 is highly biased so other choices (e.g. different in situ stress regimes or bedding plane orientations) may lead to a significant deviation from our observation.

Fig. 8
figure 8

Polar plots of critical mud pressure, BP-induced elastic anisotropy (N = 0.5, M = 2): a BP dip direction parallel to horizontal principal stress, b not parallel case where the maximum horizontal stress parallel to the north

We then rotated the azimuth of the maximum horizontal principal stress 45° clockwisely, making it not parallel with the dip azimuth, but aligned with the north. Contrary to the isotropic case (Fig. 7a), the rotation leads to a lack of plot symmetry, as shown in Fig. 8b. This is because the dip direction of the BP is no longer aligned with the horizontal principal stress: although the two borehole coordinate systems at two positions that are symmetrically located on the two sides of the magenta line (in Fig. 8b) have symmetrical compliance tensors, their two far-field stress regimes are not, leading to the different calculated pressure values and thus the plot asymmetry. If the direction of the principal stress coincides with the dip direction of BP, the polar plot restores its symmetry, as shown in Fig. 8a.

3.2 The Influence of Rock Strength Anisotropy

To study the impact of strength anisotropy only, we carried out a simulation of an elastic isotropic but strength anisotropy case. As shown in Fig. 9a, though the strength anisotropy introduced, we again observe symmetry in the plot—due to the fact that we rotated the azimuth of principal stress back to the original position so the dip direction is now parallel to the horizontal principal stress. Additionally, we see that it is safer to drill with a well trajectory perpendicular to the BP, as can be verified in Fig. 9b: when the dip angle is 90°, the horizontal well is safer in the trajectories (blue area) normal to the BP. Correspondingly, Last (1996) analysed the borehole instability in the Cusiana field using on-site observations and showed that the performance improves when drilling up-dip of the major faults and bedding while down-dip and cross-dip well trajectories are the most problematic. This is in accordance with our modelling results.

Fig. 9
figure 9

Polar plots of critical mud pressure, BP-induced strength anisotropy: a BP dip angle set to be 45°, b BP dip angle set to be 90°

To reflect the real drilling environment underground, the elastic and strength anisotropies should not be considered separately. These intrinsic rock anisotropies always coexist in a formation, and they should be coupled together, as shown in Fig. 10.

Fig. 10
figure 10

BP-induced elastic (N = 0.5, M = 2) and strength anisotropy

3.3 The Influence of Multiple Weak Planes

Figure 11a, in comparison with Fig. 8a, additionally considers the influence of NF on the constitutive relationship. We used the stiffness parameters given in Table 1 as inputs to the simulator, and we observe that an introduction of the NF stiffness based on the equivalent continuum idea increases the critical mud pressure at almost all the grids in the plot, but with different degrees. In addition, from Eq. (32), we can conclude that the higher the NF stiffness and the lower the NF density and dilation angle, the less NFs will influence the stress distribution and, subsequently, the critical mud pressure.

Fig. 11
figure 11

Polar plots of critical mud pressure, multiple weak planes: a two groups of elastic anisotropy, b two groups of both elastic and strength anisotropies

In Fig. 11b, we used all the parameters from Table 1 for calculation. Observe that due to the comprehensive consideration of the complex anisotropies, the polar plot lacks of any type of symmetry, and it is difficult to find any regularity. However, for a specific issue, we can perform a concrete analysis and derive the most accurate critical mud pressure by comprehensively considering all types of heterogeneities in the formation.

3.4 The Influence of In situ Stress and Formation Pressure

In Fig. 12a, all the three principal in situ stresses are set to be 37.17 MPa, which leads to a drop in mud pressure at almost all the places in the plot compared with Fig. 11b. Thus, the more uniform the in situ stress regime is, the greater the wellbore stability will be. In addition, applying a high formation initial pressure (22 MPa) leads to a uniform increments about 2 MPa for all the well trajectories, showing that care should be taken when drilling through a high-pressure formation (we did not attach the plot due to its exactly the same profile with Fig. 12a).

Fig. 12
figure 12

Polar plots of critical mud pressure: a homogeneous in situ stress field with a pore pressure 17.76 MPa, b poroelastic isotropy

In Fig. 12b, all the parameters are the same as those used in Fig. 12a, but we reduced the model to isotropic poroelasticity (i.e. we use Eq. (28) instead of Eq. (37) to calculate the effective stresses). Compared with its anisotropic counterpart, a dropped mud pressure of 1 MPa being observed at the two regions in the upper half circle reveals that the consideration of the poroelastic anisotropy is nontrivial.

3.5 A Correction for the Weak Plane Yield Criterion

Further, we simulated a case with the same parameters used in Fig. 9a except a decreased frictional angle of 13.6°. Figure 13a shows the calculation results: evidently, two areas with very large values are starkly positioned at around 60° inclination towards the northeast and southwest. In order to find out why this discontinuity occurs, we re-examined the flow chart (see Fig. 4). In the step ‘increase mud pressure from 0 MPa’, we set an upper limit equal to 70 MPa: if the yield criterion still cannot be satisfied when the pressure reaches the upper limit, the simulator will return a mud weight as 70 MPa for that trajectory. Therefore, in these two areas, for any pressure value inside the range 0–70 MPa, at least one equality in Eqs. (29) and (30) does not hold.

Fig. 13
figure 13

Polar plots of critical mud pressure (parameters the same with those used in Fig. 9a, except that the internal friction angle reduces to 13.6°): a before correction; b after correction

Driven by sheer curiosity, we followed the method from Liu et al. (2015) to define two yield indexes: one for intact rock [equal to the left part minus right part in Eq. (29)] and the other one for weak plane [equal to the left part minus right part in Eq. (30)]. A magnitude of yield index below zero means the rock being elastic while equalling zero indicates the onset of plastic deformation. It is therefore logical to say that the x-axis in Fig. 14 can be viewed as a yield surface. Noted that analyses of the index above zero is meaningless, since the stress fields in Eq. (23) have already changed due to the plastic flow on the yield surface.

Fig. 14
figure 14

variation of the yield index with the increase in mud pressure from 0 to 70 MPa

Figure 14 shows the variation of the maximum yield index around a borehole (from 1° to 360° intervalled by  1°) with the increase in mud pressure from 0 to 70 MPa, plotted at a trajectory with 60° inclination and 60° azimuth (see the magenta dot in Fig. 13a). For the intact rock (see the red solid line in Fig. 14), three regions can be identified: region I shows a stress regime in cylindrical coordinate by σ r  < σ 2 < σ 1, region II by σ 3 < σ r  < σ 1, and region III by σ 3 < σ 2 < σ r , where σ r is the radial principal stress acting on the wellbore while other two are the principal stresses inside the rock on a wellbore (i.e. derived from σ z , σ θ , and σ ). The red line intersects the yield surface (black straight line) at two nodes (left one: 24 MPa and right one: 48 MPa) that spans a large range, between which rocks undergo elastic deformation. These two nodes represent the pressure that satisfies the equality of Eq. (29). In contrast, when introducing strength anisotropy (see the green line), the range reduces greatly and the line (green) becomes flattened and its position moves upward. Furthermore, by reducing the internal frictional angle (ϕ) from 26.6° to 13.6°, the line (blue) locates even higher and becomes detached from the yield surface (see the inset), which indicates plastic slip along fracture will always happen: the adjustment of mud weight is thus of no use to ensure the existence of intersections (i.e. the equality of Eq. (30) never holds). This explains why the simulator returned a 70 MPa pressure, which results in the two starkly appeared areas in Fig. 13a.

In order to optimize the mud pressure for the blue line, it is assumed that its turning point (see the inset in Fig. 14) corresponds to the pressure that can minimize the plastic deformation degree around a wellbore: the pressure beyond this point will induce more severe plastic slip along the weak plane faces. Result after the correction is shown in Fig. 13b, and it presents a smoother pressure distribution.

As discussed above, this assumption does not have a valid theoretical background. To overcome the issue, one needs to get a theoretically sound analytical solution for stress/strain fields around a wellbore intercepted by fractures when plastic deformation (along the weak plane) has already occurred. However, we failed to find such a solution due to the complexity of the problem. As a result, the numerical techniques are expected for further modellings. Generally, based on the numerical simulation results, we can optimize the mud weight by minimizing either the plastic regions around a wellbore or the accumulated equivalent plastic strains on the borehole wall. It is therefore interesting to validate our assumption by the numerical results, but is beyond our scope here.

4 Conclusions

In this paper, a model that considers various types of anisotropy is established. Factors such as rock strength anisotropy, rock elastic anisotropy, multiple weak planes, in situ stress anisotropy, and poroelastic anisotropy that can have a great influence on wellbore stability are investigated. The results indicate:

  1. 1.

    (1) The anisotropies of the elastic modulus and Poisson’s ratio are both found to be beneficial to borehole stability for any type of well trajectory. (2) When considering only one type of the weak plane, it is safer to drill perpendicular to it. (3) The polar plot has symmetries if the direction of the horizontal principal stress coincides with the dip direction of weak planes. (4) The anisotropy of in situ stresses and high formation pressure are detrimental to wellbore stability. (5) The consideration of poroelastic anisotropy is nontrivial.

  2. 2.

    A drawback of the model established by Lee et al. (2012) is disclosed when the anisotropic degree of in situ stresses is high or the strength on weak planes is low. We fixed it by choosing the turning point of the yield index line as the pressure that can minimize the plastic deformation degree around a wellbore. However, the choice remains to be validated by numerical simulations.

  3. 3.

    The situation becomes complex when including multiple weak planes for two reasons: 1. many groups of discontinuity change the stress–strain relationship of rock considerably and 2. it becomes difficult to judge whether the rock on the wellbore fails through the intact rock or along the weak plane. The simulator derived can properly consider all these complexities and provide an accurate mud lower bound to ensure the safe drilling in anisotropic formations.