1 Introduction

Chaos theory is the study of the behavior of dynamical systems that are very sensitive to initial conditions. Started at first in mathematics [53, 57], then has been developed by many other research areas including physics, chemistry and biology [40, 52, 71]. Despite the fact that these dynamical systems are based on deterministic models, their high sensitivity to initial conditions or control parameters cause their outputs to be unpredictable. The beginning of the chaotic systems returns back to the 1880’s by Henry Poincare in his attempt to prove the stability of the solar system through his work on the restricted three-body problem [63]. Later, Edward Lorenz in 1961 contributed to the chaos theory [35].

One of the interesting applications of chaos is in the field of cryptography, as it concerns about techniques to secure the transfer of messages between two ends by encrypting those messages. Thus, many researchers have highlighted the strong relationship between chaos and cryptography [19, 20, 29]. More interestingly, Shannon indicated in his paper:“Communication Theory of Secrecy Systems” that was published in 1949 [56], that good transformation in good secrecy systems is achieved by basic operations that are the heart of chaotic maps [33]. In the same paper, Shannon invented two terms that are considered the major concepts of block ciphers, namely “confusion” and “diffusion”, where the mixing property and sensitivity to initial conditions of chaos generators are mapped to the diffusion and confusion of cryptosystems. Hence, the tight relationship between chaos and cryptography, created a new field of research called chaotic cryptography, where a lot of work has been published [10, 21, 22, 32, 39, 59, 65].

Chaos was used in the two types of modern encryption, which are symmetric-key encryption in its both forms, the stream ciphers and the block ciphers, and public-key encryption. In stream ciphers, the chaos generators are used to generate a stream of pseudo-random numbers in which researchers used as keys to mask the plaintext as in [37, 50, 62]. On the other hand, in block ciphers, the key is used as an initial conditions and the control parameters of the chaos generators which then generates the cipher text after many encryption rounds as the image encryption block cipher proposed in [27, 70].

This paper is mainly to improve the core mathematical model of a chaos-based cryptosystem designed by Masuda’s model [37], where many researchers have been using since 2002. Throughout the years, and after focusing on the chaos-based cryptosystems during the last decade, Masuda’s model became a discredited model as the researchers showed many weaknesses in the model such as [13]. Thus, the improvement of Masuda’s mathematical model is required to obtain a more robust model that will give us a better security level than the previous models with a faster cryptosystem. Masuda’s model was improved to give a better key space, a robust model against theoretical attacks, and a more uniformly distributed domain. Later, the cryptosystem is tested against theoretical and statistical attacks where the robustness of the proposed cryptosystem was proven by the performance and security analysis. This paper is organized as follows: Section 2 presents the literature review of the similar presented works. In Section 3, the proposed cryptosystem and contribution is described in detail. Section 4 presents the security analysis. Where Section 5 concludes the proposed work.

2 Literature review

Zhang et al., present a fast chaos based cryptosystems [72], the presented cryptosystems mainly based on Fridrich’s architecture [17], it depends on two layers, the first layer is used to achieve the diffusion effect and the second layer for confusion effect. However, Farajallah et al., present a partially cryptanalyzed method of Zhang cryptosystem [14] .

Implementation of an image encryption scheme based on the quantum logistic map is presented by Akhshani et al. [3]. The presented cryptosystem is based on one quantum logistic map using three steps: initialize the key and the plainimage, then the image is transformed into one dimensional array, finally, the map is used to encrypt the array content using the derived key. The presented cryptosystem has a good security results where the execution time is not appropriate for fast and real-time applications.

One of the most related work of the proposed cryptosystem is presented by Wang et al. [64]. In the presented cryptosystem, the both permutation and diffusion steps are combined together in order to decrease the encryption time. Moreover, the obtained security level is high for sensitive applications.

Farajallah et al. proposed a chaos based cryptosystem based on independent three chaotic maps, the obtained security results are satisfying the requirements of the real-time harvesting sensors, where as the encryption time is acceptable [15].

Kanso et al. [28] presented a chaos based cryptosystem using three dimensional chaotic maps that can defeat the sensitivity attacks. The design of the presented algorithm is simple and efficient and it achieves the required confusion and diffusion properties.

Pareek et al. [47], present a modified version of [46], in the modified version a multiple one-dimensional chaotic maps are used instead of only a one-dimensional chaotic map. The presented work divides the plaintext into variable block size, which are encrypted randomly. However, the encryption time of Pareek et al. cryptosystem is not appropriate for real-time or IoT applications.

Wong et al., [67] presented a fast chaos cryptosystem based on standard chaotic map. The structure of the presented cryptosystem consists such that the required diffusion effect is achieved in the substitution stage by simple sequential add-and-shift operations. The obtained security results and the execution time is proved the robustness and speed of this cryptosystem for secure real-time applications.

Chunhu et al., [31] presented up to date image encryption scheme based on 3D Chaotic logistic map. The first step is to generate the secret key using a modified 3D chaotic logistic map. Then use the same maps to encrypt the image with modification to improve the security level. The presented work is satisfying the security requirement. However, the execution time is not appropriate even for offline applications and it is clear the lower level of encryption throughput.

In 2019, Xu et al., [69] presents a high speed image encryption algorithm based on compressive sensing and hyperchaotic map. It is based on a new 2D sine improved logistic iterative chaotic map. The presented results show that the algorithm has high security level, good compression factor and high speed encryption. In fact, the execution time is slower than the most presented work in the literature review.

Matthews suggested the first chaotic encryption algorithm in 1989 [39]. Later, researches on chaos-based encryption increased, Baptista [4] completed one of these primary studies. A simple one-dimensional logistic map was used to encrypt each character of a text message as the integer number of iterations achieved in the logistic equation.

Jakimoski and Kocarev [26] attempted to examine Baptista cryptosystem and concluded that it has two imperfections; the encryption speed is average in correlation with other cryptosystems due to essential number of iterations, on the other hand, this system is not robust to known-plaintext attacks, however, it was the outset of using chaos theory in cryptography.

The authors of [58], present a review of image encryption techniques based on chaotic systems, the try to group the chaotic image encryption into spatial and frequency domain, where in spatial domain, the presented chaotic based cryptosystems deal with images values in normal method. However, in a frequency domain, they deal with the rate at which the pixel values are changing in a spatial domain. The presented review concludes that a high security level cryptosystem should be based on using hybrid chaotic techniques.

Fredric in 1997 [17, 18] proposed the first chaos-based image cryptography. Researchers focused more on chaotic image encryption by using different chaotic maps to overcome the traditional cryptosystem disadvantages, and it worth telling that this technique was a sufficient way for image encryption due to the speed and the strong security.

On the other hand, the authors of [5] used a one-dimensional chaotic equation alternative map that can be used for twofold image encryption with the probability of consuming a huge number of keys. Later, Z. Han et al. in [24] proposed a non-linear map which was used for duplicating pixel values. Where the authors of [1] utilized three different chaotic maps for image encryption. The authors spread a 2D cat map on 8 × 8 blocks of an image to achieve the shuffling of the pixels and used the 2D coupled logistic map to produce control parameters of shuffling. Later the shuffled image is encrypted by 1D Logistic map; thus, there was no data leakage from encrypted image.

A three encryption algorithms named as Triple-Key chaotic proposed by [61] in 2011. Those keys are an initial parameter key, 80-bit session key and control parameter key. The work was a combination between [48] and [55] which focus around the logistic chaotic map and chaotic neural network.

In 2013, the authors of [25] proposed a classical cryptosystem that focuses on the AES and the chaotic logistic map to analyze the security eligibility of both cryptosystems and to evaluate the speed of both algorithms. The AES algorithm offered a better security performance in this study, but was slower regarding the encryption running speed. On the other hand, because of the computational cost, and the easiness of implementation the logistic map is more substitute for image encryption in real-time correspondence.

Murillo-Escobar et al. in [44] presented a color image encryption algorithm based on the plain image characteristics to resist a chosen and known plain image attack, and a 1D logistic map with to get a faster encryption process based on Murillo-Escobar’s algorithm [43]. Dimensional chaotic maps have some drawbacks to be used in encryption as their data distribution is not uniform, their periodicity is relatively short, and have small key space. Thus, the authors used the 1D logistic map, as it has many powerful advantages such as the simple structure and the ease of implementation, and they are ideal for fast encryption.

Rafik et al. [41, 51] presented a privacy-preserving cryptosystem for IoT E-healthcare applications. The presented cryptosystem includes a new Pseudo Random Number Generator (PRNG). It is based on cascading the orbits of two of the 2D chaotic maps and produce the encryption keys for the cryptosystem algorithm. The presented PRNG is tested and evaluated to be used in cryptography applications.

Finally, the work in [8] is based on a chaotic system and deoxyribonucleic acid (DNA) sequence. The plain image is converted to a DNA matrix, and the chaotic map are used to generate a key matrix that is used to merge the confused DNA matrix; and then the initial values and system parameters of the chaotic system are updated by the hamming distance of the plain image and finally decoded the diffused DNA matrix, to get the ciphered image.

3 Proposed cryptosystem

3.1 Overview

The Finite State Tent Map (FSTM) mathematical model was introduced by Masuda et al. in [37, 38] as shown in (1):

$$ F_{A}(X)= \left\{\begin{array}{ll} \left\lfloor \frac{Q}{A} \times X \right\rfloor + 1 & 1\leq X< A \\ Q & X=A \\ \left\lfloor \frac{Q\times (Q-X)}{Q-A} \right\rfloor & A< X\leq Q \end{array}\right. $$
(1)

where

$$ X_{1}= \left\lfloor \frac{A\times Y}{256} \right\rfloor $$
(2)

and

$$ X_{2}= 256 - \left\lfloor (1-\frac{A}{256})\times Y \right\rfloor $$
(3)

Q here is the block size, and it is equal to 256. And \(X,A,Y \in \left \{ 1,2,3 ... Q \right \}\).

This version excluded the value 0, therefore, some authors tried to shift the model to include 0 and exclude Q from the range of X, A and Y (the plaintext, the key and the ciphertext). Still, this model had many drawbacks, such as the division by zero when A = Q, and some values of the output decreases the probability of guessing the value of the input [13]. As a result, Masuda’s model became a discredited version of the FSTM, where the last updated model was modified by [13], where \(X,Y \in \left \{ 0,1,2,3 ... Q \right \}\), \(A \in \left \{ 1,2,3 ... Q \right \}\) and Q = 256 as in (4):

$$ F_{A}(X)= \left\{\begin{array}{lr} \left\lceil \frac{Q}{A} \times (X+1) \right\rceil mod Q & 0 \leq X < A \\ \left\lfloor \frac{Q \times (Q-X) }{Q-A}\right\rfloor + 1 mod Q & A \leq X < Q \end{array}\right. $$
(4)

3.2 Proposed equation

Depending on the study of (1), and its improvement in [13], an extra part was added to the first interval of the equation to make sure that the domain will be more distributed in the proposed equation, as the domain of the output in (1) is concentrated around several values. The proposed mathematical model became as in (5)

$$ R_{Q}(X)= \left\{\begin{array}{lr} \left\lceil \frac{Q}{A(X-1)} + \frac{Q (A+1)}{X} \right\rceil mod Q & 0 \leq X < A\\ \left\lfloor \frac{\frac{Q \times (Q-X)}{1.5}}{Q-A}\right\rfloor mod Q & A \leq X < Q \end{array}\right. $$
(5)

where \(X,Y,A \in \left \{ 0,1,2,3 ... Q-1 \right \}\) and Q = 257.

The encryption quality and the information entropy are calculated for the proposed model after every single change until the best results. During the experiment, the values increased regularly till the point “\( \frac {Q (A+1)}{X}\)”, where at that point the increasing process is stopped to avoid any loss of information from the image. The middle interval of Masuda’s equation was deleted to guarantee not having the same result when A is equal to X. Those changes were done to obtain the following:

  1. 1.

    A better key space

    After studying the range of the key space in [13, 37],it is important to note that the only active bits in the key space are 66 odd values out of the 256, which makes it weak, as for some values it’s easy to guess the input value. On the other hand, the model doesn’t reach the perfect secrecy, as the key space is not equal to the plaintext space and not equal to the ciphertext space as well, which is proposed in their model to be equal to 256.

    In number theory, a and b are said to be relatively prime, or co-prime if the only positive integer that divides both of them is one. Thus, their greatest common divisor being 1 [7]. To increase the security level of the model, A and Q have to be co-prime. Pointing to the models of [13, 37], when Q = 1, with \(X,Y,A \in \left \{ 0,1,2,3 ... Q \right \}\).

    The key space will contain only: \(A =\left \{ 51, 53, 55, ... , 117, 139, 141, 143,... , 201 \right \}\) that are the cop-rime to 256 that counts 66 active bits in the key space.

    Following the same concept, Q is suggested to be 257 instead of 256 to increase the security level. Depending on the fact that 257 is a prime number, \(A \in \left \{ 0,1,2,3 ... Q-1 \right \}\) will give us 256 active bits in the keys space as all the numbers under 257 are co-prime with it, which at the same time will decrease the probability of guessing any input from the output value.

  2. 2.

    A secure model against theoretical attacks

    Masuda’s model had an interval that made it easy to make the possibilities less when guessing the input value. As the output is always 256 when X=A. Thus, by increasing the possibilities of guessing the output, the cryptosystem is considered more robust against the theoretical attacks. Depending on that point, X=A is merged to the second interval of the proposed mathematical model.

  3. 3.

    To have a more uniformly distributed domain

    The strength of the cryptosystem’s output is directly proportional to the distribution of its domain. As the main goal is to obtain a stronger core model for the chaotic systems, the model in [13] is changed to be as distributed as possible. Depending on the evaluation of the results, the proposed model gave a better distribution domain than the previous models of Masuda [37], and Farajallah [13]. The result of the information entropy analysis and the encryption quality analysis that are shown in the section of the security analysis, shows that the proposed system has a better-distributed domain than the previous models – the nearer to the uniformly distributed domain among others.

3.3 Cryptosystem design

The proposed cryptosystem is based on a hybrid encryption scheme that combines both stream and block ciphering algorithms to achieve the required security level, with a minimum encryption time. Both stream and block ciphers in cryptography belong to the family of symmetric key ciphers in which the same key is used for both of the encryption and the decryption processes.

The stream cipher converts the plaintext bits directly into the ciphertext by XORing them with pseudo-random cipher bits, while block cipher encrypts fixed size blocks that contain a group of bits from the plaintext [66]. The stream cipher has a higher speed of transformation and a low error rate, as an error that occurs in one bit will not affect the other bit. The block cipher has a high level of diffusion which any block effect will be spread into several blocks. On the other hand, the diffusion effect is low in the stream cipher, as all information of the plaintext is contained in a single ciphertext symbol. The block cipher has low encryption speed, as the entire block must be accumulated before the encryption or decryption process starts. Furthermore, the entire block may be corrupt due to an error in one bit.

In the proposed cryptosystem, the image is divided into several numbers of blocks with a block size of 256 bytes and encrypted it block by block to minimize the error bits. As shown in Fig. 1, using the Cipher-Block Chaining (CBC) mode in the proposed cryptosystems, which is a confidentiality mode as it chains (combine) the plaintext block with the previous ciphertext block. The CBC mode requires an initialization vector to combine it with the first plaintext block which is generated in the proposed system by the chaotic generator.

Fig. 1
figure 1

The proposed algorithm encryption process

Full size image

In Fig. 1, P0 represents the first plain block, the IV is the initial vector that is generated by El Assad generator [12], and C0 is the ciphered block.

The proposed algorithm encrypts the whole image using Alg2 and Alg3 shown in Fig. 1, it encrypts the odd and the even blocks using different algorithms as shown in Fig. 2.

Fig. 2
figure 2

Algorithm 1: the encryption process

Full size image

Alg2 encrypts the odd blocks based on the proposed model RQ-FSTM as shown in Fig. 3. Where as the substitution and the permutation are done in one step to decrease the encryption time. . First of all, the new position is calculated from the old one based on the proposed FSTM, then the permutation is achieved when the posn is used instead of the old position. In addition, the same equation is used in order to achieve the substitution by xoring the old value with the key. Finally, the key value is updated using the proposed FSTM but with different values of the updated process of the new position.

Fig. 3
figure 3

Algorithm 2: odd block encryption

Full size image

Alg3 encrypts the even blocks as shown in Fig. 4 using a selective substitution based on RQ-FSTM to decrease the time and to increase the encryption quality at the same time. It is similar to Alg2, but only the most 2 significant bits of the byte at the new position is xored with the 2 least significant bits of the key.

Fig. 4
figure 4

Algorithm 3: even block encryption

Full size image

Afterward, the diffusion and confusion effects in the proposed cryptosystem are transferred between blocks using the CBC mode [11]. Its worth telling as well, that the model (5) was implemented based on a look-up table to decrease the encryption time. The input of this look-up table is the generated dynamic key from the implemented version of El Assad et al. [12] chaotic generator, in addition to the byte from the plaintext.

El Assad chaotic generator was implemented to avoid the weakness in the chaotic systems regarding periodicity-generating sequences. This generator consists of two chaotic maps, i.e., the Skew Tent Map (STM) and the discrete Piece-Wise Linear Chaotic Map (PWLCM), in which are connected in parallel to generate the sequence values of 32-bit samples [16].

4 Security analysis

To design and develop a chaos-based cryptosystem, the system should be suitable and efficient for the target application, achieve the degree of the security level, and not to consume time or memory. It should offer the required security level. Analyzing the complexity of any cryptosystem is an important assessment factor. Researchers typically take this evaluation as the time of encryption/decryption, however, measure that is more comprehensive are needed to evaluate the cryptosystem. In this section, the whole security analysis for Masuda [37], Farajallah [13] and the proposed cryptosystem results are reproduced.

4.1 Theoretical analysis

4.1.1 Key space

Resisting the brute force attack needs a large secret key, with at least 128 effective and independent bits. Depending on that fact, the proposed proposed cryptosystem is robust against the brute force attack as it has a secret key with 169 bits which was calculated using two chaotic maps: the discrete STM and the discrete PWLCM [16]. And a dynamic key that consists out of 8 bits that are changeable and unique for each new block, and have been chosen out of 257 active values.

4.1.2 Ciphertext only attack

In this theoretical attack, a group of ciphertexts are available to the attackers, where they try to find the corresponding plaintexts. Where the complexity to resist such an attack is based on the available amount of the ciphertexts. This type of attack is facilitated when the attacker has multiple pieces of ciphertext generated from the same key which is not existed in the proposed model.

4.2 Differential cryptanalysis

Differential cryptanalysis is presented by Biham and Shamir for the first time to be used in Data Encryption Standard (DES) [6]. It is used in order to analyze how much a small change in the plaintext effect the corresponding ciphertext. This analysis can be used to partially or completely cryptanalyze the cryptosystem under the test. To measure this change effect two common parameters are used: The Unified Average Changing Intensity (UACI), and the Number of Pixels Change Rate (NPCR) [36, 68].

4.2.1 Plaintext sensitivity attack

Depending on the diffusion definition, any slight change in the plain image, even a change of a single bit, should statistically, change one bit out of two of the cipher image, and similarly, if one bit of the cipher image is flipped, then approximately one half of the plain image bits should change.

$$ UACI= \frac{1}{L \times C \times P \times 255} \times \sum\limits_{p=1}^{P} \sum\limits_{i=1}^{L}\sum\limits_{j=1}^{C} \left| C_{1} - C_{2} \right| \times 100 <percent> $$
(6)
$$ NPCR= \frac{1}{L \times C \times P } \times \sum\limits_{p=1}^{P} \sum\limits_{i=1}^{L}\sum\limits_{j=1}^{C} D(i,j,p) \times 100 <percent> $$
(7)

In Eqs. (6) & (7), i, j and p are the row, column, and plane indexes of the image, respectively. While L, C and P are the length, width, and plane sizes of the image. D(i,j,p) = 0 when it is the same value in C1 and C2 while it is 1 when it is different. Table 1 presents the results of the plaintext sensitivity attacks of the proposed cryptosystem for the tested images, where the optimal value for UACI is 33.46% and for NPCR is 99.61% which are given in [36, 68].

Table 1 Plaintext sensitivity tests for the proposed cryptosystem
Full size table

Two plain images are selected to be encrypted using the same secret key. While they have a difference in one bit in the first block. Most probably, the researchers chose the first bit in the image to be the different one. For more fair results, another scenario is introduced: the chosen bits will be located in the beginning, in the middle, and at the end of the first block so as to get closer results to the real application.

4.2.2 Key sensitivity attack

As well as the changes of the input, any slight change in the secret key will produce a completely different ciphered image [45], in other words, this means that any cryptosystem has to resist this attack. However, changing one bit in the key during decryption on the ciphered image will completely destroy the decryption process; the whole encryption process will fail.

Figure 5 shows the decryption process of the ciphered Lena image using the same key, but with one bit change in that key at decryption process. This confirms visually that the proposed algorithm resists the sensitivity attacks of the related used keys.

Fig. 5
figure 5

Decryption of lena image using incorrect key

Full size image

Another testing scenario of the key sensitivity which is similar to the plaintext sensitivity attacks as well: Where, one plaintext P and two secret keys with a difference of one bit. First, P is encrypted using K1 to obtain C1. Then the same P is encrypted using K2 to obtain C2. Finally, NPCR and UACI are evaluated to calculate the key sensitivity attack of the proposed cryptosystem. As shown in Table 2, the proposed cryptosystem results indicate that the proposed cryptosystem is very sensitive to a one-bit change in the secret key.

Table 2 Key sensitivity tests for the proposed cryptosystem
Full size table

5 Statistical analysis

Statistical analysis is used to measure the random behavior of any cryptosystem. In this section, the common statistical tools are used to validate the proposed algorithm.

5.1 Histogram analysis

An image histogram is a graphical demonstration that shows a visual impression of the circulation of pixels through scheming the number of pixels at each grayscale level. This graph shows the number of pixels in an image at each different intensity value.

For encrypted images, the histogram should be uniformly distributed to be strong against the statistical attacks, where cryptnalysis can benefit from the most used bit in the image and its position to reveal some information about the key [34].

The chi-square test’s result ensures whether the ciphered image pixels are uniformly distributed or not, as shown in (8) :

$$ {\chi}_{exp}^{2}= \sum\limits_{i=0}^{Q-1}\frac{(o_{i}-e_{i})}{e_{i}} $$
(8)

where Q is the number of levels (in the proposed model is 256), oi is the observed occurrence frequencies for each level in the ciphered image and ei is the expected one from the uniform distribution. In a secure cryptosystem, the experimental chi-square value have to be less than the theoretical chi-square value, which is 293 in case of α = 0.05 which is the level of significance and Q = 256, \(\chi ^{2}_{exp} < {\chi }_{th}^{2}(255,0.05)=293\).

More information on setting up the used parameter of the chi-square test can be found on the data analysis book [30].

The results in Fig. 6, show that the tested histograms are uniform and do not reveal any useful information for the statistical analysis.

Fig. 6
figure 6

Lena image 512 plain and ciphered with their Histogram

Full size image

5.2 Correlation analysis

Two neighboring pixels in a plain image are intensively corresponded in an extreme estimation of relationship coefficient of 1 and the base is 0 considered as the property of an image. On the other hand, the pixels in the encrypted image should have as low redundancy and correlation values as possible (closer to zero), even though the adjacent pixels in the plain images are very redundant and correlated [9].

To define the correlation in the encrypted images [42], the correlation coefficient (rxy) between two horizontally, vertically and diagonally neighboring pixels is calculated for 10000 randomly pairs (N) using the (9)

$$ r_{x,y} = \frac{cov(x,y)}{\sqrt{D(x)} \sqrt{D(y)}} $$
(9)

where \( cov(x,y)= \frac {1}{N}{\sum }_{i=1}^{N}(\left [ x_{i}-E(x) \right ]\left [ y_{i}-E(y) \right ])\), \(D(x)= \frac {1}{N}{\sum }_{i=1}^{N} (x-{i}-E(x))^{2}\), \(E(x)= \frac {1}{N}{\sum }_{i=1}^{N} (x_{i})\) and x,y are the pixel values of the two adjacent pixels in the tested image.

Table 3 & Fig. 7 show the correlation results for the Lena image and its corresponding cipher image, which is encrypted by the proposed cryptosystem.

Table 3 Correlation analysis of the ciphered images
Full size table
Fig. 7
figure 7

Correlation analysis of the plain and ciphered Lena image 512

Full size image

5.3 Information entropy

In any image, the values of the pixels are ranging from 0 up to 255. To have a robust algorithm for encrypting, the occurrence probability of any pixel should be almost the same. Thus, the entropy information, which is calculated using (10), will evaluate the random behavior of the encrypted message.

$$ H(C) = \sum\limits_{i=0}^{Q-1} Pro(c_{i})\times \log_{2}\frac{1}{Pro(c_{i})} $$
(10)

where H(C) is the entropy of the ciphered image C, Pro(ci) is the occurrence number of each level (i = 0, 1, 2 ... 255). In case of equal probability levels (Pro(ci) = 2− 8), the information entropy is maximal, \(H(C) = {\sum }_{i=0}^{256-1} 2^{-8} \times \log _{2} (256) = 8\) according to the above (10). Lena image statistics in the proposed cryptosystem, entropy of encrypted image using the proposed algorithm is 7.9996, which is very adjacent to the theoretical value of 8. This shows that the algorithm is secure against entropy attack.

5.4 Encryption quality

In cryptosystems, that encrypts images, pixels values as compared to the value of the same pixel before encryption, where those changes may be irregular [2]. Therefore, this means that the higher the change in the pixels values, the more effective will be the image encryption and the quality of the encryption.

Thus, the encryption quality can be defined as the total changes in pixels values between the original image and the encrypted image and the measure for the encryption quality can be the deviation between the original and encrypted image:

$$ EQ= \frac{{\sum}_{i=1}^{N}(\left| o_{i}(P) - o_{i}(C)\right| )}{256} $$
(11)

where oi(C) and oi(P) are the observed occurrences for the byte level i in the ciphered image C and in the plain image P respectively. As a result, the larger the value of EQ, the higher the level of security of the cryptosystem.

For the need of comparison, it is necessary to estimate the optimal value of EQ. The maximal value of EQ denoted as \(EQ_{\max \nolimits }\) [13], \(EQ_{\max \nolimits } = \frac {510 \times L \times C}{256^{2}}\) where L and C are the line and the column of the gray image/frame, and depending on that the ideal encryption quality is 2040. Regarding the Table 4 and Fig. 8, the proposed cryptosystem has a better EQ than the algorithms proposed by [13, 37].

Table 4 Encryption quality analysis
Full size table
Fig. 8
figure 8

Encryption quality analysis

Full size image

5.5 Complexity analysis

Calculating the complexity of the algorithm used in the cryptosystem is an important factor that determines the time of performance. On the other hand, the performance can be determined by the running speed of the algorithm or the Encryption Throughput (ET), and the number of cycles needed to encrypt one byte, which is the CPU speed in Hertz divided by the ET in bytes [13].

$$ ET = \frac{Image_{size} (Byte)}{Encryption_{time}(second)} $$
(12)
$$ Number of cycles per byte = \frac{CPU Speed_{Herts} }{ET_{Byte}} $$
(13)

The results for the encryption and decryption processes of the proposed cryptosystem are carried out using the Code::Blocks compiler of C programming on a laptop with 2.70 GHz processor Intel CoreTM i7-7500U CPU, 8GB RAM, and Windows 10, 64-bit operation system. Lena image colored with 3 different sizes ((256 × 256 × 3 bytes), (512 × 512 × 3 byte) and(1024 × 1024 × 3 byte)), Baboon and Boat images with the size (512 × 512 × 3 byte) are the image under test. Later, the results were tested again using MATLAB R2017a.

Table 5 presents the running speed of the algorithm (encryption time) in milliseconds, compared with the fastest chaos-based cryptosystems.

Table 5 Encryption time of different algorithms in millisecond
Full size table

To calculate the time performance, the average execution time for the test image after encrypting them using 1000 different secret keys is calculated. Table 6 presents the running speed of the algorithm (throughput) in megabyte per second (MBps) and the number of cycles required to encrypt or decrypt one byte. The results are compared to the fastest chaos-based cryptosystems in the literature. Through those calculations, the number of encryption rounds is identified by the required security level.

Table 6 Encryption throughput and the number of cycles for each encrypted byte - Lena image 512
Full size table

5.6 Randomness tests

To evaluate the proposed algorithm, the well-known NIST test suite are used [54], this test includes 15 test to validate the random behavior of the tested bits. To assess the proposed algorithm, the encrypted images are used as bit-stream for NIST tests to assess the behavior of the encrypted bits.

Based on [23, 54, 60], the observed results in Table 7 confirms that the random behavior of the encrypted version of Lena image 512 with more than 106 bits.

Table 7 Results of the NIST SP 800 - 22 randomness test on encrypted image - Lena image 512
Full size table

Depending on the above results, the proposed algorithm obtained a more robust model with a better security level than the previous models with a faster cryptosystem. The proposed algorithm gave a better key space, a robust model against theoretical attacks, and a more uniformly distributed domain which guaranteed having a faster cryptosystem than the existing ones in the literature, in addition, it preserved the required security level.

6 Conclusion

In recent decades, the most important communication is happening through wireless techniques using the internet to transfer data, and the main concerns remain in the subject of the security. Encryption is a unique way to guarantee the confidentiality of data.

In this paper, the problem of achieving the confidentiality of transmitted images over public channels has been studied, by using chaos-based cryptosystems. The mathematical model of a STM model was improved in this work to be used as the core structure on the proposed cryptosystem that was designed and implemented for real-time applications with a high-security level.

The obtained results confirms the high speed and high security level of the proposed solution regarding statistical tests and some known attacks, however, as a start, any chaos-based encryption algorithm should be evaluated regarding all presented attacks on the classical encryption algorithms in order to be a robust encryption algorithm [49]. The proposed algorithm also can be used in IoT and real-time applications based on the fast encryption process and high security level that has been proved in Section 5. Moreover, it can be modified as future work in order to be used as privacy preservation on the IoT when the key is not change where it is one of the main challenges of the IoT security.