1 Introduction

With the continuous development and maturity of Internet and communication technology, how to effectively protect security of data transmission and storage has become a hot topic. Encryption technology is an effective method to protect data security [1]. Among these data, digital images are widely used due to their intuitive, vivid and realistic characteristics. However, some traditional encryption methods, such as data encryption standard (DES), international data encryption algorithm (IDES) and advanced encryption standard (AES), mainly encrypt text information and are not suitable for digital images, because images have high redundancy and strong correlation between adjacent pixels [2,3,4]. Now, many scholars presented new image encryption schemes, such as using chaotic system, cellular automata, DNA sequence operations, compressive sensing and so on.

Chaotic systems are widely used in the field of image encryption because of their dynamic characteristics such as sensitivity to initial conditions, ergodicity and non-periodicity [5]. Fridrich [6] firstly proposed the permutation-diffusion framework in 1998, and then it is adopted by the vast majority of image encryption. In the permutation phase, the positions of image pixels are changed; in the diffusion stage, image pixel values are modified. Many encryption algorithms use a 1D chaotic system to design the encryption operation, for this system is easy to implement in hardware. For example, the one-dimensional Logistic mapping has been widely utilized in image encryptions [7,8,9]. However, these chaotic systems generally have shortcomings such as short periodicity and small key space, which make them easy to be cracked. High-dimensional memristive chaotic systems have more complex dynamic behavior and sensitive to initial conditions, and they are favored by image encryption researchers.

With the development of multi-disciplinary fusion, DNA computing has attracted wide attention due to its characteristics of parallel computing, large storage capacity and low power consumption [10]. Some scholars have introduced DNA sequence operations into image encryption algorithms and proposed image ciphers based on DNA computing [11,12,13,14]. In [15], utilize keystreams generated by the chaotic system to determine the DNA encoding/decoding rules of the plain image and then use the chaotic sequences to perform scrambling and DNA XOR diffusion operations on the image pixels at the same time. This encryption scheme makes the encryption process more efficient than other ciphers. However, Wu et al. [16] pointed out that this algorithm has security issues, such as the keystream generation rules have nothing to do with the plain image, and an attacker may obtain some plain image information through statistical analysis. In [17], Sun proposed a hyper-chaotic image encryption algorithm based on DNA coding and scrambling, chaotic sequences generated by a 5D hyper-chaotic system were utilized to achieve pixel-level and bit-level scrambling, and DNA XOR operations were performed for diffusion. Feng et al. [18] noted that this algorithm has a problem that the key is not related to plain image, and the attacker can obtain the equivalent diffusion matrix and scrambling matrix by selecting the plaintext attack. To enhance the relationship between the cryptosystem and plain image, SHA 256 or 512 hash values of plain images are computed to generate the chaotic keystreams for confusion and diffusion [19, 20]. However, when it comes to encrypting a huge mass of images, a large number of solving operations of chaotic systems result in low efficiency. Recently, Chen et al. [21] pointed out that some encryption algorithms [22, 23] scramble the image pixels in order have some security risks, that is, the attacker can obtain the previous information by using plaintext information. Therefore, designing secure and efficient image ciphers has become an important challenge.

As a general operation in image processing, image filtering is widely applied in image denoising, image smoothing and image sharpening. Recently, Hua et al. [24] applied the filtering operation in the image encryption for the first time, a fixed kernel was utilized to do convolution with plain image, and experimental results show that the proposed encryption algorithm can achieve good encryption results. Comparing with many existing diffusion methods, wherein, one or two previous pixels in the same row or column are often utilized to modify the current pixel in the images [15, 17], the proposed filtering method [24] adopts the surrounding pixels from different rows or columns to change the current pixel. Therefore, it may spread the change of one pixel to all the pixels faster, and then the cryptosystem may resist differential attack effectively.

Subsequently, Li et al. [25, 26] proposed a new filtering operation for image diffusion and utilized chaotic systems to generate multiple convolution kernels for filtering. However, these conventional filtering operations [24,25,26] are invalid for special images (such as All Black), resulting in lower security. For example, when a 1 × 3 mask matrix is adopted to perform a convolution operation on an all-zero image, it is assumed that the kernel of the matrix is 23, 56, 98. According to the filtering process in [24,25,26], the last element is set to 1 to make the filtering process reversible. The specific calculation process is as: 23 × 0 + 56 × 0 + 1 × 0 = 0. It can be seen from the calculation process that after the all-zero image is subjected to the filtering operation, the newly obtained image pixel value does not change, and the output result is still zero. Therefore, the filtering operation is invalid. Figure 1 gives the simulation results for some special images. Figure 1a is an image reflecting the shape of the nebula in space. After filtering, (b) is obtained, and the image information is absolutely leaked. In addition, after test image (c) is filtered, some information can still be seen visually in (d). Since the filtering operation has a low security, the encryption algorithm depends on other operations to achieve satisfactory encryption results, resulting in low encryption efficiency.

Fig. 1
figure 1

Simulation results of image filtering in [24,25,26]

Motivated by the above analyses, this paper proposes an image encryption algorithm based on image filtering, memristive chaotic systems and DNA sequence operations. The specific innovations are as follows.

  1. (1)

    A self-updating transformation based on dynamic image filtering (STDIF) is proposed. The new operation firstly performs XOR operation on the plain image and the chaotic sequence. Next, random numbers are embedded into the plain image to get a new image, and a filtering convolution operation is performed on it to modify image pixel values. This operation solves the problem that the conventional filtering operation is invalid for special image such as all-zeros and enhances the sensitivity of the encryption algorithm to plain image, and different images have different filtering operation directions.

  2. (2)

    A double random 3D matrix scrambling (DR3DMS) is presented. In most algorithms, a position index matrix is utilized to scramble the pixels of the image one by one to change its position, and this sequential scrambling method is easy for attackers to reversely get the scrambling process. To solve this problem, a new 3D matrix position mapping is constructed based on the 3D Cat mapping and orthogonal Latin cubes. It not only increases the randomness of the permutation process, but also improves the security of the encryption algorithm.

  3. (3)

    A plane diffusion of 3D DNA matrix controlled by plain image information (PD3DPI) is provided. The hash value of plain image is used to determine the diffusion plane of the 3D DNA matrix of plain image, and the plane serial number information of the 3D matrix is utilized to control the DNA operation (such as DNA XOR, addition, subtraction) of this plane, which improves the correlation between the algorithm and plain image. At the same time, since the permutation and diffusion operations are both performed on the 3D matrix, the dimension-conversion operation is removed and the encryption efficiency is improved.

  4. (4)

    A secure and efficient method for generating chaotic keystreams is proposed, which is especially suitable for encrypting large batches of images. The chaotic sequences are generated at one time by using the initial value of the chaotic system that is not related to the plain image, and then when encrypting different original images, the chaotic keystreams are selected from chaotic sequences by use of the plaintext information. It not only solves the problem of repeatedly generating chaotic sequences when encrypting different images, but also ensures that different chaotic sequences are used when encrypting different images.

The remainder of this paper is arranged as follows. Section 2 briefly provides the 4D memristive chaotic system, the 3D Cat mapping, DNA sequence operations and 3D Latin cubes. Section 3 describes the proposed image encryption and decryption methods in detail. Simulation results and security analyses are given in Sects. 4 and 5, respectively. Section 6 finally reaches a conclusion.

2 Preliminaries

This section may give the fundamental knowledge about the used 4D memristive chaotic system, 3D Cat mapping, DNA sequence operations and 3D Latin cubes.

2.1 4D Memristive chaotic system

In 2018, Karthikeyan Rajagopal et al. [27] proposed a new 4D memristive chaotic system, and it may be defined as

$$\left\{ \begin{gathered} \dot{x} = a(y - x) + {\text{byz}} + u \hfill \\ \dot{y} = {\text{cy}} - {\text{xz}}^{2} + W(u) \hfill \\ \dot{z} = - 4z + xy \hfill \\ \dot{u} = 2x - {\text{du}} \hfill \\ \end{gathered} \right.$$
(1)

where x, y, z, u are state variables, a, b, c and d are system parameters, and W(u) = e + hu2. When a = 32, b = 17, c = 15, d = 0.3, e = 4, h = 0.01 and the initial values of the system are (0.3, 0.3, 0.3, 0.3), the Lyapunov exponents calculated by the Wolf method are 3.105221, 0, −0.280985 and −24.130975, and the chaotic attractor diagram is shown in Fig. 2, which both indicates that this system is chaotic.

Fig. 2
figure 2

The chaotic attractor

2.2 3D Cat mapping

The Cat mapping, also called Arnold mapping, is a 2D mapping proposed by the mathematician Vladimir Igorevich Arnold. It is widely used in chaotic encryption. As an extension of the 2D Cat mapping, 3D Cat mapping [28] is more suitable for image encryption because of better randomness of chaotic sequences.

The mathematical expression of the 3D Cat mapping [29] is shown as

$$\left[ \begin{gathered} {\text{mx}}_{{i + 1}} \hfill \\ {\text{my}}_{{i + 1}} \hfill \\ {\text{mz}}_{{i + 1}} \hfill \\ \end{gathered} \right] = A\left[ \begin{gathered} {\text{mx}}_{i} \hfill \\ {\text{my}}_{i} \hfill \\ {\text{mz}}_{i} \hfill \\ \end{gathered} \right]\bmod N$$
(2)

The parameter matrix A can be described by

$$A = \left[ {\begin{array}{*{20}l} {1 + a_{z} a_{z} b_{y} } \hfill & {a_{z} } \hfill & {a_{y} + a_{x} a_{z} + a_{x} a_{y} a_{z} b_{y} } \hfill \\ {b_{z} + a_{z} b_{y} + a_{x} a_{z} b_{y} b_{z} } \hfill & {a_{z} b_{z} + 1} \hfill & {a_{y} a_{z} + a_{z} a_{y} a_{z} b_{y} b_{z} + a_{x} a_{z} b_{z} + a_{z} b_{y} b_{y} + a_{x} } \hfill \\ {a_{x} b_{x} b_{y} + b_{y} } \hfill & {b_{x} } \hfill & {a_{x} a_{y} b_{x} b_{y} + a_{x} b_{x} + a_{y} b_{y} + 1} \hfill \\ \end{array} } \right]$$
(3)

where ax, ay, az, bx, by and bz are system parameters, and mx0, my0 and mz0 are initial values of the 3D Cat mapping.

2.3 DNA Sequence operations

DNA sequences consist of four kinds of nucleic acid bases, they are A (adenine), T (thymine), G (guanine) and C (cytosine), where A and T are complementary, and G and C are complementary. Since 0 and 1 are complementary in the binary, 00 and 11, 01 and 10 are also complementary. Encoding 00, 11, 01 and 10 by use of A, T, G and C, there are 24 types of DNA encoding rules, but only 8 of them conform to the Watson–Crick rule [30]. Table 1 gives the encoding rules.

Table 1 DNA coding rules

As for the DNA encoding and decoding of images, DNA encoding is to convert pixels in digital images into DNA sequences according to certain DNA coding rules. DNA decoding is the reverse process of DNA encoding, and the corresponding pixel value can be recovered through it. In a digital image with a gray level of 256, one pixel can be converted into an eight-bit binary string, and each two-bit binary string can be represented by a DNA base.

For example, a pixel grayscale value of the image is 57, which can be expressed in binary number as 0011101. Suppose that the DNA sequence is firstly encoded according to rule 2, the resulting DNA sequence is ATCG. When it is decoded according to rule 8, the binary number is 10011100. In this way, by adopting the different encoding and decoding rules, performing DNA encoding and decoding on image pixels is also an effective encryption method by changing image pixel values. After the pixels of images are transformed to DNA sequences, they can be subjected to the DNA addition, DNA subtraction and DNA XOR operation shown in Table 2.

Table 2 DNA operations

2.4 3D Latin cubes

A 3D Latin cube of order L = (ai,j,k) is a cube of size l × l × l (l rows, l columns and l order), with only l different elements, and each element appears exactly once in each row and each column of each order. The existence and construction methods of orthogonal Latin cubes have been studied and described in Refs. [31, 32]. Figure 3 shows the process of constructing a 3D orthogonal Latin square using a cube with the size of 3 × 3 × 3.

Fig. 3
figure 3

The construction of an orthogonal Latin cube

Figure 3a is an orthogonal Latin cube L constructed from three different elements named by L0, L1 and L2; (b) is a cube M composed of a set of three elements corresponding to positions L0, L1 and L2 at positions M0, M1 and M2; (c) shows the process of constructing the corresponding position cubes M0, M1 and M2 from the first row of the orthogonal Latin squares L0, L1 and L2.

The 3D orthogonal Latin cube Lz1 may be expressed by

$$L_{{Z1}} (i,j,k) = M(M_{0} (i,j,k),M_{1} (i,j,k),M_{2} (i,j,k))$$
(4)

where 0 ≤ i, j, k ≤ lr (lr is the side length of the cube).

Recently, Xu et al. [33] proposed a new image encryption algorithm based on the 3D bit matrix and Latin cube. The orthogonal Latin cube is generated by chaotic sequences, and then the 3D bit matrix of the plain image is scrambled and diffused. The experimental results show that using 3D orthogonal Latin squares to achieve spatial scrambling of 3D matrix elements can achieve a good scrambling effect, but it is invalid for special images (such as All Black and All White), which reduces the encryption efficiency of the algorithm. To solve this problem, we propose a self-updating transformation operation before image scrambling to change the pixel values of the image.

3 The proposed image cipher

In this section, Sect. 3.1 gives the generation and selection method of the chaotic keystreams, Sect. 3.2 describes the self-updating transformation based on dynamic image filtering (STDIF), the method of converting the 2D plain image into 3D DNA matrix is given in Sect. 3.3, generation of parameters and initial values of 3D Cat mapping are presented in Sect. 3.4, double random 3D matrix scrambling (DR3DMS) is introduced in Sect. 3.5, plane diffusion of 3D DNA matrix controlled by plain image information (PD3DPI) is described in Sect. 3.6, and the complete image encryption algorithm is shown in Sect. 3.7.

3.1 The generation and selection of chaotic keystreams

In the chaos-based image encryption algorithm, chaotic sequences generated by the chaotic systems are usually employed as the keystreams for the scrambling and diffusion stages. In order to enhance the resistance to plaintext attacks, the information of plain image is generally utilized to perturb the generation of chaotic sequences, so that different chaotic sequences can be used to encrypt different plain images, ensuring the security of the encryption algorithm. However, when a large number of images need to be encrypted, the generation of chaotic sequences will waste a lot of time, and thus the cryptosystem is not efficient. In order to settle this problem, the initial values and system parameters of chaotic systems unrelated with the plain image are utilized to generate the chaotic sequences at one time. And then, the hash values of plain images are utilized to select different chaotic sequence fragments among them. This method of generating the keystream not only solves the problem of repeatedly generating chaotic sequences when encrypting different images, but also ensures that different chaotic sequences are picked for different plain images, which enhances the correlation between the cryptosystem and plain image. The detailed generation process of the chaotic keystreams is as follows:

Firstly, calculate the SHA 256 hash value hv of the plain image, and it is composed of hv1, hv2, …, hv256. Subsequently, as shown in Fig. 4, divide the hv into eight parts on average, where each 32 elements form a group, and then convert them to the respective decimal numbers, name them as k1, k2, k3, k4, k5, k6, k7, k8. Here, k1, k2, k3, k4 are used to pick up chaotic keystreams, and k5, k6, k7 and k8 are applied in other encryption processes.

Fig. 4
figure 4

The dividing of SHA 256 hash value hv

Assuming that the plain image is P(M × N), the generation and selection process of chaotic keystream are described as follows.

Step 1 Iterate the 4D memristive chaotic system N0 (N0 ≥ 2500 + 4MN) times with the initial values x0, y0, z0 and u0, discard the previous t (t ≥ 500) values to avoid transient effects and obtain the chaotic sequences X, Y, Z, U.

Step 2 Compute the parameters T1, T2, T3 and T4 by

$$\left\{ \begin{gathered} T_{1} = 500 + k_{1} \bmod 400 \hfill \\ T_{2} = 500 + k_{2} \bmod 600 \hfill \\ T_{3} = 500 + k_{3} \bmod 800 \hfill \\ T_{4} = 500 + k_{4} \bmod 1000 \hfill \\ \end{gathered} \right.$$
(5)

Step 3 Among chaotic sequences X, Y, Z and U, select the elements from T1 + 1 to T1 + 4MN, from T2 + 1 to T2 + 4MN, from T3 + 1 to T3 + 4MN and from T4 + 1 to T4 + 4MN, obtain new chaotic sequences and denote them as X1, Y1, Z1, U1.

As illustrated from the generation process of chaotic keystreams, chaotic sequences are produced by the hash value of the plain image, hash value will change greatly for just a little alternation of plain image, and thus different plain images will obtain different chaotic sequences. Therefore, the proposed image encryption algorithm may withstand known-plaintext and chosen-plaintext attacks.

3.2 Self-updating transformation based on dynamic image filtering (STDIF)

In order to solve the conventional filtering operation is invalid for special image such as all-zeros, a self-updating transformation based on dynamic image filtering (STDIF) is designed combining plain image information with chaotic system, which is used to transform the pixel values of plain image for the first time. The specific operation is as follows:

Step 1 Use the 4D memristive chaotic system to generate a two-dimensional matrix W sized of M × N and then perform XOR operation on the plain image P(M × N) and W to obtain the new image P1 according to

$$P_{1} (i,j) = {\text{XOR}}[W(i,j),P(i,j)]$$
(6)

where \(i \in [1,M]\), and \(j \in [1,N]\).

Step 2 Determine the direction of the filtering operation. Compute parameter Dir by use of

$${\text{Dir}} = 1 + k_{5} \bmod 4$$
(7)

where k5 is a part of the hash value of plain image, and 0 ≤ k5 ≤ 233–1. When Dir = 1, 2, 3 and 4, add the random numbers generated by the chaotic system to the image P1 and then obtain the new image P2.

The specific generation process of the image P2 is as follows.

Case 1 When Dir = 1, add two column random numbers of size M to the left of image P1, so that the image size becomes M × (N + 2). As shown in Fig. 5a, following the arrow direction in the figure, the image is filtered from left to right.

Fig. 5
figure 5

Schematic diagram of random number embedding and filtering operations

Case 2 When Dir = 2, put two column random numbers of size M to the right of image P1, so that the image size becomes M × (N + 2). As shown in Fig. 5b, following the arrow direction in the figure, the image is filtered from right to left.

Case 3 When Dir = 3, place two row random numbers of size N on the upper side of image P1, so that the image size changes to (M + 2) × N. As shown in Fig. 5c, following the arrow direction in the figure, the image is filtered from top to bottom.

Case 4 When Dir = 4, add two row random numbers of size N on the lower side of image P1, so that the image size becomes (M + 2) × N. As shown in Fig. 5d, following the arrow direction in the figure, the image is filtered from bottom to top.

Step 3 Use the chaotic sequence X1 gotten in Sect. 3.1 to generate the matrix W2. The specific process is as follows.

Step A Perform a quantization operation on the chaotic sequence X1 to obtain a new sequence X2 by use of

$$X_{2} (n) = \bmod [{\text{floor}}(X_{1} (n) \times 10^{{14}} ),256],\quad n \in [0,4{\text{MN}} - 1]$$
(8)

Step B Randomly select 3MN numbers from the sequence X2, record them as X3 and then convert sequence X3 into 1 × 3 or 3 × 1 matrix W1. Finally, set the value of the last three-digit number in the matrix W1 to 1, and get a new matrix W2.

Step 4 Perform a filtering operation between matrix W2 and the image P2 according to the filtering direction in step 2. After all pixels have been diffused, the added random numbers are removed to obtain a new image P3. The dynamic filtering is finished.

In order to make our STDIF express more clearly, take an example to illustrate it. Suppose that the plain image is P(M × N), the image after adding random numbers is P2(M × (N + 2)), and the chaotic matrix used for filtering is W2(1 × 3MN). Two pixels in the plain image P are selected as an example for illustration, and the detailed process is shown in Fig. 6.

Fig. 6
figure 6

Illustration of our dynamic image filtering process

Firstly, two 1 × 3 filtering matrices are selected from a filter matrix W2. Then, the filtering operation is performed on the image P2. The operation process is as, mod (78 × 14 + 156 × 32 + 1 × 65, 256) = 5, and mod (12 × 32 + 9 × 65 + 1 × 71, 256) = 16, and then one gets two pixels of the filtered image. The filtering operation of other pixels in the image is the same as the above-mentioned operation. After all pixels have been done, the new matrix is gotten by getting rid of two column random numbers. The filtering kernel participating in the operation is generated by the chaotic system, and thus the filtering diffusion process is dynamic and random.

3.3 Converting the 2D plain image into 3D DNA matrix

The 2D plain image after the self-updating transformation is transformed to a 3D DNA matrix for the sake of the subsequent 3D DNA level scrambling and diffusion. Figure 7 provides the process of converting a 2D plain image into a 3D DNA matrix. The specific steps are as follows.

Fig. 7
figure 7

Flowchart of converting a 2D image into a 3D DNA matrix

Step 1 Convert the self-updated image P3(M × N) into a 1D binary sequence P4 with the size of 1 × 8MN, and then perform DNA encoding on sequence P4 to get the DNA sequence P5 sized of 1 × 4MN according to one DNA encoding rule.

Step 2 Calculate the parameter \(l = \sqrt[3]{{1 \times 4{\text{MN}}}}\). If the result is a positive integer, go to step 6, otherwise go to step 3.

Step 3 Define a number R = r3 (r is a positive integer). Find the largest R that satisfies the condition R < S when S = M × N × 4, S = n × r3 = n × R. Then, the image is divided into n cubes.

Step 4 According to the obtained n and r, construct n cubes with the size of r × r × r, and then skip to step 7.

Step 5 If none of the above conditions are met, and a cube cannot be constructed directly. Thus, add zero elements to plain image until the above conditions are met, and then skip to step 1.

Step 6 According to the value of l calculated in step 2, build a cube with a size of l × l × l.

Step 7 Output the constructed cube, name it as P6 and record the side length of the cube as lr.

According to the above steps, the plain image of any size can be converted into a 3D DNA matrix. During the conversion process, the following situations may exist.

Case 1 The size of the image can be directly converted into a 3D DNA matrix.

For example, a grayscale image with the size of 256 × 256 can be converted into a DNA matrix with the size of 64 × 64 × 64. The detailed process is as follows. Firstly, convert a grayscale image sized of 256 × 256 into a 1D binary sequence with the size of 1 × (256 × 256 × 8), and then turn it into a DNA sequence with the size of 1 × (256 × 256 × 4) according to the DNA encoding rules. Next, calculate the l value and l = 64, and construct a 3D DNA matrix with the size of 64 × 64 × 64.

Case 2 The image needs to be segmented to convert it into multiple 3D DNA matrices.

For example, a grayscale image with the size of 512 × 512 can be transformed into four DNA matrices with the size of 64 × 64 × 64. The process is as follows. Firstly, convert a grayscale image with the size of 512 × 512 into a 1D binary sequence with the size of 1 × (512 × 512 × 8), and then turn it into a DNA sequence with the size of 1 × (512 × 512 × 4). Next, calculate n and r (n = 4, r = 64), and construct four 3D DNA matrices with the size of 64 × 64 × 64.

Case 3 The image size cannot meet the above conditions, and a zero-padded operation is required to convert it into a 3D DNA matrix.

For example, a 1 × 1 grayscale image cannot be directly converted into a 3D DNA matrix. And then, convert it into a 1D binary sequence with the size of 1 × 8, and then turn into a DNA sequence with the size of 1 × 4. It cannot be converted into a 3D DNA matrix. Add zeros on the image, make it become a 1D binary sequence with the size of 1 × 16, and then convert it to a DNA sequence with the size of 1 × 8. After that, turn it into a 3D DNA matrix sized of 2 × 2 × 2.

3.4 Generation of parameters and initial values of 3D Cat mapping

To enhance the relationship between the cryptosystem and plain image, the system parameters ax, ay, az, bx, by and bz and initial values mx0, my0 and mz0 of the 3D Cat mapping are determined by the hash value of plain image and chaotic system. Specifically, k1-k6 is used to calculate the initial values mx0, my0 and mz0, k7 is utilized to compute the parameters ax, ay and az, and k8 is applied to determine the parameters bx, by and bz. The specific process is as follows:

Firstly, mx0, my0 and mz0 are gotten by

$$\left\{ \begin{gathered} {\text{mx}}_{0} = x_{{((k_{1} + k_{4} )\bmod 450)}} \times 10^{{13}} \bmod 32 \hfill \\ {\text{my}}_{0} = y_{{((k_{2} + k_{5} )\bmod 450)}} \times 10^{{13}} \bmod 32 \hfill \\ {\text{mz}}_{0} = z_{{((k_{3} + k_{6} )\bmod 450)}} \times 10^{{13}} \bmod 32 \hfill \\ \end{gathered} \right.$$
(9)

Next, when k7mod3 = 0, ax, ay and az are obtained by

$$\left\{ \begin{gathered} a_{x} = x_{{(k_{1} \bmod 100)}} \times 10^{{10}} \bmod 16 \hfill \\ a_{y} = x_{{(k_{2} \bmod 100)}} \times 10^{{10}} \bmod 32 \hfill \\ a_{z} = x_{{(k_{3} \bmod 100)}} \times 10^{{10}} \bmod 64 \hfill \\ \end{gathered} \right.$$
(10)

When k7mod3 = 1, ax, ay and az are gotten by

$$\left\{ \begin{gathered} a_{x} = y_{{(k_{1} \bmod 200)}} \times 10^{{11}} \bmod 16 \hfill \\ a_{y} = y_{{(k_{2} \bmod 200)}} \times 10^{{11}} \bmod 32 \hfill \\ a_{z} = y_{{(k_{3} \bmod 200)}} \times 10^{{11}} \bmod 64 \hfill \\ \end{gathered} \right.$$
(11)

When k7mod3 = 2, ax, ay and az are obtained by

$$\left\{ \begin{gathered} a_{x} = z_{{(k_{1} \bmod 300)}} \times 10^{{12}} \bmod 16 \hfill \\ a_{y} = z_{{(k_{2} \bmod 300)}} \times 10^{{12}} \bmod 32 \hfill \\ a_{z} = z_{{(k_{3} \bmod 300)}} \times 10^{{12}} \bmod 64 \hfill \\ \end{gathered} \right.$$
(12)

When k8mod3 = 0, bx, by and bz are computed by

$$\left\{ \begin{gathered} b_{x} = x_{{(k_{4} \bmod 150)}} \times 10^{9} \bmod 64 \hfill \\ b_{y} = x_{{(k_{5} \bmod 150)}} \times 10^{9} \bmod 32 \hfill \\ b_{z} = x_{{(k_{6} \bmod 150)}} \times 10^{9} \bmod 16 \hfill \\ \end{gathered} \right.$$
(13)

When k8mod3 = 1, bx, by and bz are obtained by

$$\left\{ \begin{gathered} b_{x} = y_{{(k_{4} \bmod 250)}} \times 10^{{11}} \bmod 64 \hfill \\ b_{y} = y_{{(k_{5} \bmod 250)}} \times 10^{{11}} \bmod 32 \hfill \\ b_{z} = y_{{(k_{6} \bmod 250)}} \times 10^{{11}} \bmod 16 \hfill \\ \end{gathered} \right.$$
(14)

When k8mod3 = 2, bx, by and bz are calculated by

$$\left\{ \begin{gathered} b_{x} = z_{{(k_{4} \bmod 350)}} \times 10^{{13}} \bmod 64 \hfill \\ b_{y} = z_{{(k_{5} \bmod 350)}} \times 10^{{13}} \bmod 32 \hfill \\ b_{z} = z_{{(k_{6} \bmod 350)}} \times 10^{{13}} \bmod 16 \hfill \\ \end{gathered} \right.$$
(15)

3.5 Double random 3D matrix scrambling (DR3DMS)

Figure 8 gives the ordinary 3D matrix scrambling, and vector H is used to shuffle every pixel in the 3D matrix of plain image one by one. When vector H is found, the plain image is easily gotten. Recently, Zhang et al. [34] proposed a new scrambling method combining Chen chaotic system and 3D Cat mapping, but it was quickly discovered by Wu et al. [35] that the encryption algorithm’s keystreams and parameters of 3D cat mapping are completely dependent on the Chen chaotic system. Thus, the key parameters are the same for all test images, and they are easily cracked by the chosen-plaintext attack. To solve this problem, combining the 3D orthogonal Latin square with 3D Cat mapping, this paper constructs a new double random 3D matrix scrambling (DR3DMS), where the parameters of the 3D Cat mapping are generated by plain image information, and different original images will generate different position mapping vectors. Figure 9 displays the diagram of DR3DMS.

Fig. 8
figure 8

Ordinary 3D matrix scrambling

Fig. 9
figure 9

Diagram of DR3DMS

Firstly, the chaotic sequence Z1 is utilized to construct a 3D orthogonal Latin cube Lz1, and the position sequence Lz1(i, j, k) is obtained. Suppose that the position sequence of the 3D DNA matrix DP of plain image is (i, j, k), and then a mapping relationship is established according to the position sequence of the DNA matrix and Latin square, and it is shown by

$$(i,j,k) \to (L_{{z1}}^{i} ,L_{{z1}}^{j} ,L_{{z1}}^{k} )$$
(16)

where 1 ≤ i ≤ lr, 1 ≤ j ≤ lr, 1 ≤ k ≤ lr, (i, j, k) represents the position of the element in the 3D matrix, and \((L_{{z1}}^{i} ,L_{{z1}}^{j} ,L_{{z1}}^{k} )\) corresponds to a random position in the orthogonal Latin square.

Secondly, a position sequence Mp(i, j, k) is obtained by the 3D Cat mapping, and then a corresponding relationship is established between the position sequence of the 3D DNA matrix DP and that of the Cat mapping and illustrated as

$$(i,j,k) \to (M_{p}^{i} ,M_{p}^{j} ,M_{p}^{k} )$$
(17)

where 1 ≤ i ≤ lr, 1 ≤ j ≤ lr, 1 ≤ k ≤ lr, (i, j, k) represents the position of the element in the 3D DNA matrix, i, j and k correspond to mx, my and mz in Eq. (2), respectively, and \((M_{p}^{i} ,M_{p}^{j} ,M_{p}^{k} )\) are their random positions in the 3D Cat mapping.

Based on Eqs. (16) and (17), a new mapping of random position sequences to random position sequences can be obtained, and that is

$$(L_{{z1}}^{i} ,L_{{z1}}^{j} ,L_{{z1}}^{k} ) \to (M_{p}^{i} ,M_{p}^{j} ,M_{p}^{k} )$$
(18)

An example of two-pixel shuffling is given to illustrate the DR3DMS. Assume that the first element of a 3D Latin square is (4, 9, 46) and the second element is (7, 6, 32); the first element generated by the 3D Cat mapping is (6, 25, 35), and the second element is (36, 24, 16). When manipulate DR3DMS on the 3D matrix of the plain image, move the elements in position (4, 9, 46) to position (6, 25, 35), and move the elements in position (7, 6, 32) to (36, 24, 16). When the attacker obtains the position of one element, the position information of the previous element cannot be derived in reverse. The security level of confusion effect is enhanced.

3.6 Plane diffusion of 3D DNA matrix controlled by plain image information (PD3DPI)

Image encryption in Refs. [34, 35] involves multiple matrix dimension transformations, the original image is firstly transformed from a 2D matrix to a 3D matrix to complete scrambling and then converted from a 3D matrix to a 2D matrix and then to a 1D matrix to attain the diffusion operation, finally, it is transformed to a 2D matrix to complete the encryption operation. For these encryption methods, the larger the image is, the longer the encryption time is, and accordingly, the encryption efficiency of the algorithm is reduced.

In our cipher, the confusion and diffusion operation of plain image are both manipulated on the 3D matrix, which reduces the conversion of different matrix dimensions and shortens the encryption time. In addition, the diffusion process is highly related to the plain image. The choice of the diffusion plane is controlled by the hash value of the test image. The DNA operation method (including DNA addition, subtraction, XOR) also depends on it. Different from most current image encryption algorithms based on DNA sequences [7, 9, 11, 12], wherein the chaotic system is used to generate a key matrix, and then it and the plain image are performed a single DNA addition, or DNA subtraction, or DNA XOR operation to diffuse the image. In our plane diffusion of 3D DNA matrix controlled by plain image information (PD3DPI), different planes of the 3D matrix of the plain image use different DNA calculation methods, which makes the algorithm difficult to crack. The specific process is described as follows.

Firstly, a DNA mask matrix is generated. Encode the chaotic sequence U1 to generate a 2D DNA mask matrix U2 by use of

$$U_{2} = {\text{DNA}}\_{\text{enc}}(\bmod [{\text{floor}}(U_{1} \times 10^{{14}} ),4],{\text{De}})$$
(19)

where \({\text{DNA}}\_{\text{enc}}\)(a, b) means performing the DNA encoding operation on the elements of the sequence a by the DNA encoding rule b, and De is the encoding rule. The encoding rule of the mask matrix is the same as that of the plain image.

Secondly, the diffusion plane is selected by using the hash value of plain image. Specifically, the parameter Pm is computed to select the plane matrix Q in the 3D matrix by use of

$${\text{Pm}} = 1 + (k_{1} + k_{3} + k_{5} )\bmod 3$$
(20)

when the parameter Pm = 1, 2 and 3, the 2D planar DNA matrix Q corresponds to the plane x–y, the plane x–z and the plane y–z of the 3D matrix, respectively.

Finally, manipulate DNA diffusion on the 2D planar matrix Q (lr × lr) of the 3D matrix P7 by use of

$${\text{Pd}} = 1 + {\text{ce}}\bmod 3$$
(21)
$$P_{8} = \left\{ {\begin{array}{*{20}l} {{\text{DNA}}(Q + U_{2} ),\quad {\text{Pd}} = 1} \hfill \\ {{\text{DNA}}(Q - U_{2} ),\quad {\text{Pd}} = 2} \hfill \\ {{\text{DNA}}(Q \oplus U_{2} ),\quad {\text{Pd}} = 3} \hfill \\ \end{array} } \right.$$
(22)

where ce (1 ≤ ce ≤ lr) represents the plane number information of the 3D matrix of plain image. When the parameters Pd = 1, 2 and 3, the DNA mask matrix U2 and the 2D planar DNA matrix Q are, respectively, subjected to the DNA addition, subtraction and XOR operation. After performing DNA operations on all planes, a diffused matrix P8 is obtained.

Figure 10 illustrates the process of our PD3DPI. In this figure, the 3D DNA matrix P7 represents a scrambled 3D matrix, and the DNA plane matrix Q denotes a plane on the matrix P7. The DNA mask plane U2 and plane matrix Q performed DNA operations (DNA addition, DNA subtraction, DNA exclusive OR). After all the plane in the 3D matrix P7 are completed, a 3D matrix after diffusion is obtained.

Fig. 10
figure 10

The diagram of PD3DPI

In the proposed PD3DPI, 2D diffusion planes in the 3D matrix are selected to conduct the diffusion operation by use of the hash value of plain image, and then different DNA sequence operations are manipulated on the diffusion plane and key matrix to achieve encryption, which strengthens the sensitivity of the cipher to plain image information.

3.7 The complete image encryption algorithm

Figure 11 gives the schematic diagram of our cipher. The plain image P sized of M × N is firstly subjected to STDIF, and the resulting image is DNA dynamic encoded to a DNA matrix. Subsequently, the DNA matrix is converted to a 3D DNA matrix, scrambled by DR3DMS and diffused by PD3DPI. Finally, the cipher image C is obtained by the DNA decoding. Moreover, the 256-bit hash value of the original image is obtained, and the whole encryption operations are controlled by the hash value and the 4D memristive chaotic system.

Fig. 11
figure 11

The diagram of the proposed image encryption algorithm

As illustrated in Fig. 12, the detailed encryption steps are as follows:

Fig. 12
figure 12

The diagram of the detailed encryption process

Step 1 Generate chaotic sequences. Bring initial values (x0, y0, z0, u0) into the 4D memristive chaotic system to generate chaotic sequences X, Y, Z, U, calculate the hash value hv of the plain image P(M × N) and then bring it into Eq. (4) to obtain new chaotic sequences X1, Y1, Z1, U1. The specific steps are shown in Sect. 3.1.

Step 2 Use the chaotic sequence X1 and hash value hv to perform the STDIF on the plain image. Firstly, perform XOR operation on the plain image P with the chaotic sequence to obtain P1. Then, a random number sequence is added to the plain image of the size of M × N to obtain an image P2 of the size that satisfies the filtering operation. Finally, the resulting image is filtered and the new image P3 is obtained. The specific steps are described in Sect. 3.2.

Step 3 Perform the DNA dynamic encoding on the image P3. Firstly, the DNA encoding rule sequence De is generated via

$${\text{De}} = 1 + \bmod [{\text{floor}}(Y_{1} \times 10^{{10}} + k_{4} ),8]$$
(23)

Then, the pixels of the image P3 are converted into a 1D binary sequence P4 having the size of 1 × 8MN. Next, P4 is DNA encoded by use of the sequence De to obtain the sequence P5 sized of 1 × 4MN. Here, the plain image information is used to generate the DNA encoding rules, ensuring that different plain images have different DNA encoding rules.

Step 4 As demonstrated in Sect. 3.3, the DNA sequence P5 is transformed into a 3D DNA matrix P6 (lr × lr × lr).

Step 5 Firstly, produce a 3D Latin cube by using the chaotic sequence Z. Then, as described in Sect. 3.4, calculate the initial values and parameters of the 3D Cat mapping through the hash value of the plain image, and generate the 3D Cat mapping sequences. Next, as illustrated in Sect. 3.5, perform DR3DMS on the matrix P6 to obtain a scrambled matrix P7.

Step 6 According to Sect. 3.6, use the DNA encoding rule De to encode the selected chaotic sequence U1 to obtain the key matrix U2, and then perform PD3DPI on the matrix P7 to obtain matrix P8.

Step 7 Manipulate the DNA decoding on the matrix P8, and then transform it into a 2D matrix to obtain an encrypted image C. Here, the DNA decoding rule matrix \(Dc\) can be obtained by

$${\text{Dc}} = 1 + \left( {8 - {\text{De}}} \right)$$
(24)

If the image is divided into multiple 3D matrices, each matrix is encrypted to obtain the corresponding cipher image according to the above encryption operations. Finally, each cipher image is merged into the final cipher image.

The image decryption operation is the inverse of the encryption one. The initial values x0, y0, z0, u0 of the chaotic system and the SHA 256 hash value need to be transmitted to the receiver as a key. The image decryption process is as follows. Firstly, generate the chaotic sequences required by the decryption algorithm, and then sequentially perform inverse DNA decoding, inverse PD3DPI, inverse DR3DMS on the cipher image, and then perform inverse DNA encoding operation and inverse STDIF on the resulting image to get the plain image.

4 Experimental results

In this section, simulation results will be given to prove the security and effectiveness of the proposed encryption algorithm. The experimental environment is as follows: CPU: Intel Core i5-8300 H, 2.30 GHz; Memory: 8 GB; Operating system: Windows 10; Coding tool: Matlab 2016a. Table 3 gives the initial values and parameters of the chaotic system and the initial iteration parameter t required to run the encryption algorithm. Table 4 shows the SHA 256 hash value of the test image. The images are encrypted and decrypted, and the results are shown in Fig. 13.

Table 3 Experimental parameters
Table 4 SHA 256 hash value of the test images
Fig. 13
figure 13figure 13

Experimental results

In Fig. 13, the first column is the plain image with the size of 256 × 256. From top to bottom are “All Black,” “All White,” “Airfield,” “Baboon,” “Dollar,” “Finger” and “Lena,” “Peppers,” the second column is the self-updated image, the third column is the cipher image and the fourth column is the corresponding decrypted image. It can be seen from the experimental results in Fig. 13 that the cipher image is similar to a noisy image, and people cannot visually obtain any information of the plain image. At the same time, it can be observed that there is no difference between the decrypted image and the plain image.

5 Security analyses

In this section, the proposed image encryption method may be assessed from the following aspects: key space, histogram analysis, correlation of adjacent pixels, information entropy, differential attack, key sensitivity analysis, noise attack analysis, time complexity analysis and comparison with other studies.

5.1 Key space

Generally speaking, good image encryption algorithms need to be able to resist various brute force attacks or exhaustive attacks. The larger the key space of the encryption algorithm is, the better the ability to resist brute force attacks is. The keys in the proposed algorithm include: (1) the initial values of the chaotic system x0, y0, z0, u0; (2) the 256-bit hash key generated by the SHA 256 function of the plain image and the initial iteration parameter t. If the computer precision is 10–14, our key space is at least 104 × 14 = 1056 > 2168. Table 5 lists the comparison results on the key space with other image encryption algorithms. The data show that the key space of the proposed algorithm is larger than that of Refs. [36,37,38,39], which shows that the proposed cipher has a larger key space and is sufficient to resist various types of brute force attacks.

Table 5 Comparison of key space

5.2 Histogram analysis

The histogram can visually display the distribution of the gray value of the test image [24, 39], and its mathematical expression is shown by

$$h(k) = \frac{{n_{k} }}{N},\quad k = 0,1,...,L - 1$$
(25)

where N represents the number of all pixels in the image, nk represents the number of the kth gray level of the image and L represents the gray level of the image. Generally, it is required that the gray histogram of cipher images should be flat and uniform.

The grayscale images “Lena,” “Peppers” are utilized as the test objects, and the histograms of the plain images and their corresponding cipher images are shown in Fig. 14. As observed from Fig. 14 that the histogram distribution of the plain image is uneven and undulating, however, that of the cipher image is uniform and flat, and the plain image information cannot be obtained by analyzing the cipher image.

Fig. 14
figure 14

Histograms of the plain images and cipher images

In addition, the histogram variance and chi-square test are used to quantitatively assess the uniformity of the cipher images. The smaller the histogram variance of the image is, the higher the uniformity of the image gray value is. The histogram variance is calculated by [40]

$$\text{var} (Z){\text{ = }}\frac{1}{{n^{2} }}\sum\limits_{{i = 1}}^{n} {\sum\limits_{{j = 1}}^{n} {\frac{1}{2}} } \left( {z_{i} - z_{j} } \right)^{2}$$
(26)

where Z = {z1, z2, …, z256} is the frequency of pixel occurrence, and zi and zj are the numbers of pixels with gray values equal to i and j, respectively.

The chi-square is computed by [3]

$$\chi _{{{\text{test}}}}^{2} = \sum\limits_{{i = 1}}^{k} {\frac{{(o_{i} - e_{i} )}}{{e_{i} }}}$$
(27)

where k is gray level (k = 256 in this test), oi and ei are gray values of test and expected frequency values, respectively. When the significance level is 0.05, \(\chi _{{{\text{0}}{\text{.05}}}}^{{\text{2}}}\) = 293.25 [41]. If \(\chi _{{{\text{test}}}}^{2} < 293.25\), it means that the data have passed chi-square test.

The test results of the histogram variance are shown in Table 6. By comparing the variances of the plain image, it can be seen that the variances of the cipher image are significantly reduced. The chi-square test results are listed in Table 7, which indicates that the cipher images all have passed the chi-square test. In this manner, the distribution of cipher images gotten by our cipher has good randomness, and our encryption can effectively resist statistical attacks.

Table 6 Histogram variance of plain images and cipher images
Table 7 Chi-square test results of cipher images

5.3 Correlation of adjacent pixels

Visually meaningful images have high correlation between pixels and their neighboring pixels in horizontal, vertical and diagonal directions, which facilitates cryptanalysis. Then, a good cipher should reduce or even remove the correlation to resist against statistical analysis [3, 20, 21]. To test the correlation between adjacent pixels, we randomly select 15,000 pairs of adjacent pixels from the plain images and the corresponding cipher images and calculate the correlation coefficients in the horizontal, vertical and diagonal directions. The correlation coefficients may be computed by [30]

$$R_{{x,y}} = {\text{COV(}}x,y)/\sqrt {D(x)D(y)}$$
(28)
$${\text{Cov}}(x,y) = \frac{1}{N}\sum\limits_{{i = 1}}^{N} {(x_{i} - E(x))(y_{i} - E(y))}$$
(29)
$$E(x) = \frac{1}{N}\sum\limits_{{i = 1}}^{N} {x_{i} }$$
(30)
$$D(x) = \frac{1}{N}\sum\limits_{{i = 1}}^{N} {(x_{i} - E(x))^{2} }$$
(31)

where x and y, respectively, represent the gray values of adjacent pixels in the image, N represents the total number of pixel pairs, E(x) and D(x) are the respective mean and variance of x, Cov(x, y) represents the covariance of x and y, and Rx,y represents the correlation coefficient of x and y.

Figure 15 illustrates the correlation distribution of horizontally, vertically and diagonally adjacent pixels in the plain image Lena and its cipher image. As can be seen from Fig. 15, the adjacent pixels of the plain image in the horizontal, vertical and diagonal directions are concentrated on the diagonal line, but the cipher image has uniform pixel distribution in the interval [0, 255]. Thus, the correlations of adjacent pixels in the cipher image have been effectively reduced.

Fig. 15
figure 15

Correlation distribution of adjacent pixels in plain image Lena and its cipher image

Table 8 displays the results of correlation coefficients of adjacent pixels in the plain images and cipher images. It is observed from Table 8 that the correlation coefficient of the plain images is close to 1, which indicates that there is a strong correlation between adjacent pixels of the plain images in all directions. While correlation coefficients of the corresponding cipher images approach 0, which means that the correlation between adjacent pixels of the cipher images has been effectively broken and indicates that the proposed encryption algorithm can effectively withstand statistical analysis attacks.

Table 8 Correlation coefficients of the plain images and cipher images

5.4 Information entropy

Information entropy reflects the randomness and unpredictability of test information. The information entropy H (m) of the information source m can be defined as [3]

$$H(m) = \sum\limits_{{i = 0}}^{{2^{n} - 1}} {p(m_{i} )\log \frac{1}{{p(m_{i} )}}}$$
(32)

where p(mi) is the probability of mi. For a cipher image with 256 Gy levels, the theoretical value of information entropy is H(m) = 8. If the entropy of the cipher image approaches to 8, it means that the performance of cipher image is more random, and the probability of information leakage is less.

Table 9 lists the information entropy of the plain images and cipher images produced by our cryptosystem. As can be seen from Table 9, the entropy values of all cipher images are above 7.99, which are close to the theoretical value of 8 and illustrate the strong randomness of the cipher images. Therefore, the proposed image encryption is secure against the entropy attack.

Table 9 Information entropy of the plain images and cipher images

5.5 Differential attack

Generally, the number of pixel change rate (NPCR) and the unified average changing intensity (UACI) are applied to measure the change degree of the cipher image when the value of one pixel in plaintext changes. The higher their values are, the stronger the algorithm’s ability to resist differential attacks is. They may be computed by [24, 30]

$${\text{NPCR}} = \frac{{\sum\nolimits_{{{\text{ij}}}} {D(i,j)} }}{{W \times H}} \times 100\%$$
(33)
$${\text{UACI}} = \frac{1}{{W \times H}} \left[ {\sum\limits_{{{\text{ij}}}} {\frac{{\left| {C_{1} (i,j) - C_{2} (i,j)} \right|}}{{255}}} } \right] \times 100\%$$
(34)

where W and H are the width and height of the image, C1 represents the encrypted image, C2 is the cipher image obtained by changing the plain image, and D represents a matrix of the same size as the cipher image. The value of D(i, j) is determined by C1(i, j) and C2(i, j). If C1(i, j) = C2(i, j), then D(i, j) = 0, otherwise D(i, j) = 1.The theoretical value of NPCR is 99.60%, and the theoretical value of UACI is 33.46% [42]. The test results are closer to the theoretical values, and the algorithm’s ability is stronger to resist differential attacks.

In the test, the original image is encrypted to a cipher image. Then, one pixel in the plain image is changed slightly, and another cipher image is obtained by encrypting it using the same keys. Finally, NPCR and UACI values between these two cipher images are computed and illustrated in Table 10. As can be seen from Table 10, the NPCR values are close to 99.60% and the UACI values are close to 33.46%, which means that the proposed method is sensitive to the small change in the plain image and may withstand the differential attack well.

Table 10 NPCR and UACI results for different images

5.6 Key sensitivity analysis

A good image encryption algorithm is very sensitive to the key in encryption process and decryption process. In what follows, the key sensitivity analyses in encryption and decryption phases are both analyzed.

5.6.1 Key sensitivity analysis in encryption phase

This section uses a slightly different key from the correct key to encrypt the plain image Airfield (shown in Fig. 16a) to test the sensitivity of the key in the encryption process. The correct key set K0 contains the key parameters x0, y0, z0, u0 and the SHA 256 hash value of the plain image. First, the plain image is encrypted with K0 to obtain a cipher image (shown in Fig. 16b). Then, change the key parameters x0, y0, z0, u0 by △ (△ = 10–14), only one parameter is changed at each time, and keep other parameters unchanged to obtain the key sets K1, K2, K3, K4. Next, use the obtained K1- K4 sequentially to encrypt the plain image to obtain the corresponding cipher images, as shown in Fig. 16c–f. Figure 16g–j shows the differential images between the correct cipher image Fig. 16b and wrong cipher images Fig. 16c–f.

Fig. 16
figure 16

Key sensitivity results in encryption process. a The plain image Airfield, b The cipher image, c Encrypted image with K1, d Decrypted image with K2, e Decrypted image with K3, f Decrypted image with K4, g The differential image between (c) and (b), h The differential image between (d) and (b), i The differential image between (e) and (b), (i) The differential image between (f) and (b)

In addition, the NPCR values between the correct and wrong cipher images are quantitatively tested and shown in Table 11. It can be seen that when the key undergoes a slight change, the pixel difference ratio between cipher images by the correct key set and slightly different key set is close to 99.60%, which indicates that the proposed algorithm is very sensitive to the key in encryption process.

Table 11 NPCR values between the correct and wrong cipher images

5.6.2 Key sensitivity analysis in decryption phase

This section utilizes the obtained key sets (K1, K2, K3, K4) to decrypt the cipher image Airfield (shown in Fig. 16 (b)). The corresponding decrypted images are shown in Fig. 17. Table 14 lists the NPCR values between the decrypted image and the plain image (shown in Fig. 16a). It can be seen clearly that the obtained decrypted image is noise-like with the slightly changed key, and no valid information can be obtained from them. Moreover, the test results in Table 12 are all greater than 99.60%, indicating that our algorithm is very sensitive to the secret key in the decryption phase.

Fig. 17
figure 17

Key sensitivity results in decryption process. a Decrypted image with K1, b Decrypted image with K2, c Decrypted image with K3, d Decrypted image with K4, e The differential image between (a) and Fig. 16a, f The differential image between (b) and Fig. 16a, g The differential image between (c) and Fig. 16a, h The differential image between (d) and Fig. 16a

Table 12 NPCR values between the decrypted images and plain image

5.7 Noise attack analysis

In order to test the anti-noise attack ability of our cryptosystem, the salt and pepper noise (SPN), speckle noise (SN) and Gaussian noise (GN) are added to the cipher images one by one, and then decrypt them with the correct key, and the corresponding decryption results are shown in Fig. 18. Table 13 shows the PSNR values between the corresponding decrypted images and plain image.

Fig. 18
figure 18

Noise attack results

Table 13 PSNR values between the decrypted images and plain image

From the results in Fig. 18 and Table 13, one may observe that

  1. (1)

    The image encryption algorithm proposed in this paper has the best resistance to the SPN attacks. When the noise intensity increases from 0.0001 to 0.0007, the PSNR values between the decrypted image and the plain image change from 34.8909 to 27.0216 dB.

  2. (2)

    The encryption algorithm has a certain resistance to the SN attack. When the noise intensity varies from 0.0001 to 0.0007, the PSNR values between the decrypted image and the plain image change from 18.4651 to 14.9483 dB.

  3. (3)

    The resistance of the proposed image encryption algorithm to the GN attack is general. The decrypted images are obtained after adding the Gaussian noise pollution of 0.00001 to 0.00007, and the approximate outline of the plain image can be watched visually.

In summary, the proposed cryptosystem is robust to different noise attacks.

5.8 Time complexity analysis

Apart from security analyses, time complexity is also an important assessing parameter [3], and it is related with all the encryption processes. It is assumed that the size of the plain image is M × N, and there are MN pixels in the original image. The first step in the encryption process is to generate chaotic sequences, the time-consuming part is an iterative calculation of the memristive chaotic system to obtain four floating-point chaotic sequences, and its time complexity is O(4 × M × N).

In the second encryption step, the STDIF is performed on the pixel level. Firstly, the time-consuming operation is performing the XOR operation, and the time complexity is O(M × N). Then, the direction of the filtering operation is determined by the plaintext information, and its time complexity is O(1). The time complexity of generating the quantized filtering matrix is O(3 × M × N). The last consuming-time part is the use of the filtering matrix to perform filtering operation on the image, and the time complexity is O(M × N). All in all, the time complexity of Step 2 is O(3 × M × N).

In the third encryption step, the time complexity of the DNA dynamic encoding of the plain image is O(8 × M × N). In the fourth encryption step, the 2D DNA matrix is converted into a 3D matrix, and its time complexity is O(lr × lr × lr). In Step 5, the DR3DMS is performed. First, the memristive chaotic system is used to generate the 3D orthogonal cube, and the time complexity is O(8 × M × N). Then, a 3D Cat mapping is used to generate a 3D matrix, and the time complexity is O(lr × lr × lr). Finally, scramble the 3D DNA matrix, and its time complexity is O(lr × lr × lr). In Step 6, the time complexity of the PD3DPI is O(lr × lr × lr). In Step 7, the image needs to be performed DNA decoding, and its time complexity is O(8 × M × N).

Since the computational complexity of the proposed image encryption algorithm depends on the highest time complexity of all steps, the time complexity of our encryption scheme is O(8 × M × N).

5.9 Resisting known-plaintext and chosen-plaintext attacks

Many image encryption methods have been attacked by known-plaintext and chosen-plaintext attacks [43,44,45]. In the proposed cipher, some strategies have been adopted to enhance the security of the algorithm.

Firstly, the used chaotic sequences depend on the plain image. Specifically, the chaotic sequences are obtained at one time by iterating the chaotic system with the external key, and then when encrypting different original images, the different chaotic keystreams are chosen from the obtained chaotic sequences according to the plaintext information.

Secondly, in STDIF and DR3DMS, the used chaotic sequences are highly sensitive to the plain image. 3D orthogonal Latin cube is constructed by the resulting chaotic sequence. And the initial values and system parameters of the used 3D Cat mapping are computed by the hash value of the plain image.

Thirdly, the diffusion process has the relation with the plain image. We use the hash value of plain image to get the diffusion plane of the 3D DNA matrix of plain image, and the plane serial number information of the 3D matrix is used to determine the corresponding DNA operation (such as DNA XOR, addition, subtraction) of this plane.

Conclusively, some methods have been utilized to improve the plaintext relation in the proposed algorithm, so that our cipher may withstand known-plaintext and chosen-plaintext attacks effectively.

5.10 Comparison with other studies

In order to assess the performance of the proposed image cryptosystem, this paper chooses the grayscale image “Lena” (256 × 256) to compare the correlation coefficients of adjacent pixels, NPCR/UACI, information entropy and time complexity with other algorithms, and these results are listed in Tables 14, 15, 16, 17 and 18.

Table 14 Comparing results of correlation coefficients
Table 15 Comparison of NPCR/UACI for the Lena by different encryption algorithms
Table 16 Comparison of information entropy with other algorithms
Table 17 Performance comparisons with others studies for Lena sized of 256 × 256
Table 18 Comparison of time complexity between different algorithms

It can be seen from Table 14 that the correlation coefficient of adjacent pixels of the cipher image obtained by our algorithm is lower than those in Refs. [16, 25, 46,47,48,49,50,51], which shows that the proposed encryption algorithm effectively reduces the correlation between adjacent pixels. Besides, it can be seen from Table 15 that the NPCR and UACI of our method are close to the theoretical values 99.60% and 33.46%, which are less than Ref. [52], better than those in Refs. [47, 53, 54] and just same with Ref. [28].

Moreover, as can be seen from Table 16 that the information entropy value of the cipher image obtained by the proposed encryption is close to the theoretical value of 8, which is better than Refs. [42, 53, 55, 56] and less than Ref. [57]. Besides, as can be seen from Table 17, our algorithm has the more satisfactory correlation coefficients, NPCR an UACI than those in Refs. [55, 58,59,60], the gotten information entropy values by our algorithm are larger than those in Refs. [55, 59, 60] and less than that in Ref. [58].

Additionally, Table 18 shows that the time complexity of our cryptosystem is the same as that of Ref. [61], which is slightly better than those of Refs. [62, 63]. In summary, the cipher image obtained by the proposed encryption algorithm has high security and good efficiency.

6 Conclusion

In this paper, a new image encryption algorithm is proposed by combining image filtering, memristive chaotic systems and DNA sequence operations. STDIF is applied for modifying the pixel values of the plain image. Based on the 3D Latin squares and 3D Cat mapping, DR3DMS is presented to randomly move the elements of the 3D matrix of plain image to other places. PD3DPI is used to diffuse the permutated 3D matrix of the original image. Besides, the DNA encoding rules of plain image are related to the plain image and chaotic system, ensuring that each element corresponds to an encoding rule, and the DNA decoding rules can be directly generated based on the encoding rules to reduce the calculation amount, simplifying the generation of DNA decoding rules and improving the encryption efficiency.

Simulation results and security analyses show that the proposed encryption scheme has a large key space, high key sensitivity, a certain ability to resist various attacks and can be applied in the field of image security. In the proposed cipher, the plain image is encrypted to a noisy cipher image, so that we may not find any information from the cipher image, and it may be transmitted securely over the open Internet. But when the user wants to retrieve an image, he or she must decrypt all the cipher images to the corresponding plain images, thus a lot of time and resources are wasted. In the future work, we will explore the new image encryption method which considers the image privacy protection and usability.