1 Introduction

The development of Internet technology enables digital multimedia to be spread in real-time via Internet. In the digital world, even non-professional users can easily process digital images with tools such as Photoshop, Assetizr and Picasa. Due to the various methods of image destruction, it is often difficult to protect copyrights. Digital watermarking has been recognized as a promising approach for ensuring the authenticity and integrity of images. It is an art of hiding secret information (the watermark) within digital data, such that the embedded watermark can be identified or extracted to confirm the validity of the data [17]. The secret information may be text, image, audio, or video type of contents [19].

1.1 Related work

Watermarking technology can be divided into robust watermarking and fragile watermarking. Robust watermarking is mainly for copyright insurance. Fragile watermarking is sensitive to a broad variety of bends [17]. However, most watermarking schemes embed single watermark with single purpose, which have great limitations in practical applications. Generally, in many applications such as copyright protection, data integrity, tamper detection, fingerprint recognition, and broadcast monitoring, a multi-watermarking scheme is needed.Some scholars have proposed to embed two robust watermarks at the same time to achieve better copyright protection. Singh et al. [22] presents lifting wavelet transform (LWT) and discrete cosine transform (DCT) based robust watermarking approach for telehealth applications, which hides the signature watermark and patient report into the host medical image. Kumar et al. [12] presents a dual watermarking technique using discrete wavelet transform (DWT), singular value decomposition (SVD), and SPIHT (set partitioning in hierarchical trees). It embeds two encoded watermarks into the transformed host image. Su et al. [23] proposed a new blind watermarking algorithm. By modifying the direct current (DC) coefficient in DCT, the watermark is embedded four times to achieve better robustness. Zhang and Darwish et al. [3, 31] respectively combined with particle swarm optimization (PSO) and adaptive genetic algorithm to balance both watermarking robustness and imperceptibility.The above schemes embed two watermarks only for robustness. In order to obtain more functions in a scheme, many scholars have paid attention to multi-watermarking with both robustness and fragility. Hurrah et al. [8] embeds a robust watermark using an efficient inter-block coefficient differencing algorithm. The authentication of the content has been ensured by embedding a fragile watermark in the spatial domain. Kamili et al. [9] applies DCT for robust watermark embedding. Then it uses prediction error extension to embed the fragile watermark in the spatial domain. Ahmadi et al. [1] embeds a robust watermark into the blue channel of RGB color space based on DWT, HVS and SVD domains with a specialized PSO optimization to balance the trade-off between robustness and imperceptibility. A fragile watermark is embedded into all channels of RGB color space. Duan et al. [4] embeds a robust watermark into the two layers of R and G based on transformations such as NSST and DWT, and then it embeds the hash value calculated by the two layers of R and G as a fragile watermark into the B layer.Although multi-watermarking can simultaneously meet the requirements of copyright protection and malicious tampering of multimedia content, the above algorithms will lead to irreversible distortion of the image. Therefore, reversible watermarking has attracted scholars’ attention in recent years. The advantage of reversible watermarking is that if there is no attack, the watermarked image can be completely restored to the original one. These types of watermarks can be used in fields with higher requirements for images, such as medicine and military. Tian et al. [25] first proposed the concept of Difference Expansion (DE) in 2003, DE is used for watermark embedding, a new reversible data embedding method is proposed, but the embedding capacity of the embedding scheme is not large. In order to increase the embedding capacity, Lee et al. [13] uses pixel correlation to change the scan mode and adjust the block size to increase the maximum embedding capacity of the original block from 3 bits to 8 bits. Wang et al. [27] introduced a reversible data hiding algorithm based on two-way differential expansion, which expands the difference between two adjacent pixels in two directions, and embeds information in the left pixel. Girdhar et al. [6] proposed a reversible watermarking algorithm to embed information by moving the difference between the vertices in the 3D mesh model. On the basis of DE, Li et al. [14] put forward a pixel value ordering (PVO) method. The maximum and minimum values of each block are predicted by other pixels of the block according to their pixel value orders. Zhou et al. [35] and Fan et al. [5] respectively proposed a novel PVO with changeable step size (CPVO) algorithm and an improved PVO (IPVO) algorithm to improve the image quality.In recent years, reversible watermarking algorithms based on interpolation are introduced to scale the image and embed the watermark into the sampling points. Kaw et al. [10] proposed a new large-capacity and reversible watermarking algorithm based on pixel repetition, which uses the best pixel repetition rate to securely embed the watermark into the image. On this basis, Parah et al. [18] proposed a new reversible watermarking scheme based on the pixel repetition method (PRM) and modular algorithm, which can provide high embedding capacity while maintaining good imperceptibility. Hassan et al. [7] chose the existing enhanced neighbor mean interpolation (ENMI) and modified neighbor mean interpolation (MNMI) technology to scale-up the original image before embedding the secret data. This method achieves good image quality under high embedding capacity.The above schemes all implement the reversibility of watermarking, but their schemes are fragile to common attacks. To solve this problem, reversible robust watermarking scheme is satisfied with the demand. Wang et al. [29] proposed a new spatially-based robust steganography algorithm based on the significant bit difference expansion (SBDE) method, which increases the embedding capacity, but the distortion caused by invalid embedding is relatively high. Kumar et al. [11] improved the algorithm of Wang et al. [29] and proposed a new robust reversible data hiding (RRDH) scheme based on double-layer embedding. The redundancy of HSB plane elements is effectively used, and the distortion caused by invalid embedding is reduced. To further improve robustness, [20] and [28] apply wavelet transforms and histogram-based methods to embed reversible robust watermarks. Roy et al. [20] present a robust reversible image watermarking scheme based on DCT and histogram shifting. By modifying AC coefficient, a binary watermark is embedded in each transform block and the location map is embedded into the cover image by histogram shift technology. Wang et al. [28] applied a high pass filter to each block to generate a histogram which is a Laplacian-like distribution. The watermark is embedded into the blocks by shifting the generated histogram. Experimental results show that the above schemes are robust to common attacks with reversibility.In recent years, some adaptive algorithms [33, 34, 36] and deep learning algorithms [30] have been applied to digital watermarking. Adaptive schemes are put forward to determine the embedding position of the watermark. [16, 32] proposed adaptive watermarking schemes using integer wavelet transform. Zhang et al. [32] and Meng et al. [16] respectively use HVS and element relations to adaptively select the position of the embedded watermark to achieve better invisibility. Ansari et al. [2] Use artificial bee colony (ABC) to control the embedding strength of the watermark to achieve a balance of invisibility and robustness.

1.2 Contributions

In summary, although reversible robust watermark and fragile watermark have achieved fruitful results. A reversible robustness and fragility multi-watermark scheme for color images has not been proposed. The main research work of this paper is as follows:

  1. (1)

    In this paper, a comprehensive reversible watermarking scheme of robustness and fragility for color images is proposed for the first time. This paper uses integer wavelet transform and differential histogram shift scheme to achieve reversible robust watermarking, and improve the existing PEE method to embed fragile watermarking. Both robust and fragile watermarks can be embedded into color images at the same time. After extracting the watermark, the watermarked Image can be restored, and the recovered Image is exactly the same as the original host Image.

  2. (2)

    In this paper, integer wavelet transform and differential histogram shift method are combined to achieve a robust watermarking scheme. In order to make watermarked image more invisible, this paper proposes a pre-embedding scheme to select the optimal embedding position and generate the local map. The R and G layers of the color image use the same local map to reduce the local map storage capacity. See Section 3.1 for details.

  3. (3)

    In this paper, the traditional PEE method is improved. A prediction error length map (PELM) was first proposed, instead of using local map to record the embedding position of watermark, we record the length map. For the position of too large prediction error (the prediction error length is too large), we will not embed watermarks to avoid large distortions. During the generation of tamper detection graph, a map correction scheme for tamper detection is proposed to improve the extraction accuracy. In this scheme, the blocks are divided into three classes, tampered blocks, suspected blocks and untampered blocks. After the initial tamper detection graph is generated, the proposed correction scheme is used to judge the suspicious block twice to improve the tamper detection accuracy. See Section 3.2 for details.

The rest of the paper is arranged as follows. Section 2 introduces background knowledge, integer wavelet transform, difference histogram shift, and we proposed prediction error length map(PELM). Section 3 describes the embedding and extraction process of the watermark in detail. Section 4 makes experimental results of the proposed method. Finally, a summary is made in Section 5.

2 Algorithm basis

2.1 Integer wavelet transform

IWT (Integer wavelet transform) was proposed and proved by Sweldens et al. [24] in 1996. It presents the lifting scheme, a simple construction of second-generation wavelets. Compared with DWT, IWT can improve computational efficiency and achieve lossless image reconstruction.

Figure 1 shows the decomposition and reconstruction process of lifting wavelet transform. Lifting wavelet transform is divided into three steps, splitting, predicting and updating. After lifting scheme, the input signal sj can be decomposed into a low-frequency part sj− 1 and a high-frequency part dj− 1. For the low-frequency subset sj− 1, perform the same splitting, predicting, and updating to further decomposed sj− 1 into sj− 2 and dj− 2,⋯.In this way, after n times of decomposition, the wavelet of the original data sj is represented as {sj−n,dj−n,dj−n+ 1,⋯ ,dj− 1}. sj−n represents the low-frequency part, {dj−n,dj−n+ 1,⋯ ,dj− 1} is the series of the high-frequency part from low to high. The specific steps are as follows:

Fig. 1
figure 1

Lifting wavelet transform process

Full size image
Split :

is to divide the original signal sj = {sj,k} into two disjoint subsets. The length of each subset is half of the original signal. It is common to divide a sequence into even sequences ej− 1,k = {sj,2k} and odd sequences oj− 1,k = {sj,2k+ 1}, that is, Splitsj = (ej− 1,oj− 1).

Predict :

takes advantage of the correlation between even and odd sequences. One sequence (usually an even sequence ej− 1) is used to predict the other sequence (usually an odd sequence oj− 1). The difference dj− 1 between the actual value oj− 1 and the predicted value p(ej− 1) reflects the degree of approximation between the two, which is called the wavelet coefficient. Generally speaking, the more relevant the data, the smaller the wavelet coefficient. If the prediction is reasonable, the dataset dj− 1 contains much less information than the original subset oj− 1. The prediction process is as follows: dj− 1,k = oj− 1,k − pk(ej− 1). Where pk can take the ej data itself, which is pk(ej− 1,k) = ej− 1,k or take the average value of the adjacent data, \(p_{k}(e_{j-1})=\frac {e_{j-1,k}+e_{j-1,k+1}}{2}\).

After the splitting step, the global features (such as the mean) of the subset may be inconsistent with the original data. To maintain these characteristics of the original data, an Update process is essential. The process is as follows: sj− 1 = ej− 1 + U(dj− 1). Where sj− 1 is the low-frequency part of sj. The update operator can also take different functions, such as \(U_{k}(d_{j-1})=\frac {d_{j-1,k}}{2}\), \(U_{k}(d_{j-1})=\frac {d_{j-1,k-1}+d_{j-1,k}}{4}+\frac {1}{2}\).

2.2 Difference histogram shift

Due to the correlation between adjacent audio information, Liang et al. [15] realized audio robust and reversible watermarking using high-order differential histogram shift. In the image, there is a correlation between pixels. After the color image is divided into three layers of R, G, and B, any layer also meets this characteristic. On this basis, this paper proposes a differential histogram shift robust watermarking scheme for an image. Divide the embedded area into N blocks (\(N=\frac {H}{n} \times \frac {W}{n}\), H is the length of the host image, W is the width of the host image, and n is the size of each sub-block). That is, each block has n × n pixels, which is defined as blockl, where l is the block number (l = 1,2,3⋯N). In one block, half of pixels are weighted as 1, and the others are -1. Thus, each group will get a difference value D(l), the calculation formula is as Formula (1).

$$ D(l)=\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n} (-1)^{i+j} \times block^{l}(i,j) $$
(1)

Use this method to calculate the difference of N blocks. Then generate a prediction error histogram, as shown on the left side of Fig. 2.

Fig. 2
figure 2

Histogram of prediction error and embedded histogram

Full size image

Suppose the offset of the histogram is B, B = T + G. T is the absolute value of the maximum prediction error, G is the embedding strength. If B < T, the watermark cannot be completely extracted. Shift the histogram to embed the watermark. If the watermark w(i) = 1, use Formula (2) for shifting, and if w(i) = 0, no operation is performed. The difference histogram after embedding is shown on the right in Fig. 2.

$$ D(l)^{\prime}=D(l)+\frac{D(l)}{|D(l)|} \times B \qquad \qquad if\quad w(i)=1 $$
(2)

The change of the block difference value D(l) is achieved by shifting the pixels in the l block. To ensure that the change of the pixel value is an integer, the offset of each pixel is set to \(\beta (k)=\lfloor \frac {B+(k-1)}{M}\rfloor ,M=n \times n,k=1,2,3 {\cdots } M\), change the blockl(i,j) as Formula (3).

$$ block^{l}(i,j)^{\prime}=block^{l}(i,j)+(-1)^{i+j} \times \frac{D(l)}{|D(l)|} \times \upbeta(k) $$
(3)

When extracting the watermark, only the D(l) of the watermark image needs to be calculated. Use Formula (4) to extract the watermark, and the grouped image is shown on the right in Fig. 2.

$$ w=\left\{ \begin{array}{lr} 0 \qquad D(l) \in class2 \\ 1 \qquad D(l) \in class1,3 \end{array} \right. $$
(4)

Similarly, according to the range of D(l), reversible recovery is performed. When restoring, use Formula (1) to calculate the \(D(l)^{\prime }\) of the watermark image, and use Formula (5) to restore.

$$ D(l)^{r}=D(l)^{\prime}-\frac{D(l)^{\prime}}{|D(l)^{\prime}|} \times B \qquad \qquad if \quad w(i)=1 $$
(5)

To realize the change of D(l), the group l in the watermark image is restored using Formula (6).

$$ block^{l}(i,j)^{r}=block^{l}(i,j)^{\prime}-(-1)^{i+j} \times \frac{D(l)^{\prime}}{|D(l)^{\prime}|} \times \upbeta(k) $$
(6)

2.3 Prediction error length map

The prediction error length map (PELM) is proposed to replace the local map in the traditional PEE method. Figure 3 is an 8 × 8 block from the image, which is used as an example for illustration. The blue block pixels in the figure are used to predict the green block pixels, so that each 8 × 8 block has 18 prediction positions, which can be embedded in the way of PEE.

Fig. 3
figure 3

Map of the predicted error of a block

Full size image

This paper use the method of diamond prediction [21] to predict the green block. The following is a detailed explanation of the embedding, extraction and recovery process of a pixel.

The prediction and embedding formula is shown in Formula (7-10).

First, use a 3 × 3 block size to predict the central pixel value C. Pc is the average of the four pixels above and below the center pixel, Pe is the prediction error, as in Formula (7).

$$ Pe=C-Pc $$
(7)

w is the watermark sequence, and only the position where w(i) = 1 is embedded. Use Formula (8) to calculate the prediction error value after embedding the watermark \(Pe^{\prime }\).

$$ Pe^{\prime}=\left\{ \begin{array}{lr} Pe \qquad w(i)=0 \\ \frac{Pe}{|Pe|}\times(2\times|Pe|+1) \qquad w(i)=1 \end{array} \right. $$
(8)

Modify the center pixel, the modified center pixel value is \(C^{\prime }\), as in Formula (9).

$$ C^{\prime}=Pc+Pe^{\prime} $$
(9)

Calculate the prediction error value length l and store it.

$$ l=\lceil log_{2}(Pe)\rceil $$
(10)

The watermarking extraction formula is shown in Formula (11-13). First calculate the predicted value of the pixel after the embedded watermark, \(C^{\prime }\) is the central pixel value of the watermarked image, \(Pc^{\prime }\) is the predicted central pixel value of the embedded watermark, and \(Pe^{\prime }\) is the calculated prediction error value.

$$ Pe^{\prime}=C^{\prime}-Pc^{\prime} $$
(11)

Calculate the prediction error value length of the watermark map \(l^{\prime }\).

$$ l^{\prime}=\lceil log_{2}(Pe^{\prime})\rceil $$
(12)

The watermark extraction method is as formula (13).

$$ w(i)=\left\{ \begin{array}{lr} 0 \qquad l=l^{\prime} \\ 1 \qquad l \neq l^{\prime} \end{array} \right. $$
(13)

After the watermark is extracted, the center pixel is restored, and Per is the prediction error after restoration.

$$ Pe^{r}=\left\{ \begin{array}{lr} Pe^{\prime} \qquad w(i)=0 \\ \frac{Pe}{|Pe|}\times \frac{|Pe|}{2} \qquad w(i)=1 \end{array} \right. $$
(14)

Obtain the restored center pixel value Cr according to Per

$$ C^{r}=Pc^{\prime}+Pe^{r} $$
(15)

3 Proposed watermarking scheme

In this paper, the robust watermark is embedded into R layer and G layer by using integral wavelet transform and differential histogram shift. Then, the embedded \(\text {R}^{\prime }\) and \(\text {G}^{\prime }\) are used to generate the hash sequence. Hash sequence, as fragile watermark, is embedded into B layer by PEE. Figure 4 shows the flow chart of watermark embedding.

Fig. 4
figure 4

The flow chart of watermark embedding

Full size image

In the procedure of extracting the watermark, extract the fragile watermark first, the hash sequence H1 is generated by the \(\text {R}^{\prime }\) and \(\text {G}^{\prime }\) layers. Then, the hash sequence H2 in the \(\text {B}^{\prime }\) layer is extracted by the predictive error length map. The tamper detection map is generated by comparing H1 and H2. Secondly, extract the robust watermark use IWT and difference histogram shift. Finally, decide whether the suspected image is tampered according to tamper detection graph. If yes, the tamper detection graph will be displayed. Otherwise, the watermarked image will be recovered. Figure 5 shows the flow chart of watermark extraction procedure.

Fig. 5
figure 5

The flow chart of watermark extraction

Full size image

3.1 Copyright protection: robust watermarking scheme

3.1.1 Robust watermark embedding

Robust watermark embedding is divided into two stages, the first stage is the pre-embedding, and the second stage is watermark embedding. The pre-embedding stage is mainly used to determine the embedding position and generate the local map. In this paper, the R layer and the B layer of the host image are first divided into 16 × 16 non-overlapping blocks, each block is transformed by 3-IWT, and the 2 × 2 low frequency area is selected to calculate the difference value D(l). This paper uses the method of sorting the difference value to determine the final embedding position. The smaller the difference value is, the better the invisibility of the watermarked image. Figure 6 is a heat map generated from the differential value calculated after the block transformation of the R layer, G layer and B layer of Lena. The closer the color in the heat map is to blue, the smaller the difference is, and the closer to green, the greater the difference is. In this paper, the R and the B layer use the same local map for the reason that the distribution of the heat maps of the three layers is similar, which can also be seen from Fig. 6. In the embedding stage, the local map generated in the pre-embedding stage is used to embed the watermark. The specific embedding process of the watermark is as follows. Steps 1-4 are the pre-embedding process, and 5-7 are the embedding process.

  1. Step1.

    Perform Arnold scrambling on the watermark image and generate a binary watermark sequence w, the watermark sequence length is lenw.

  2. Step2.

    Divide layer R and layer G into non-overlapping 16 × 16 blocks and calculate the difference using Formula (1). Use Formula (16) and (17) to get the sequence SR and SG. Add the corresponding positions of the two sequences to generate the sequence S, as in the Formula (18). Sort the sequences from small to large Ssort = Sort(S).

    $$ S_{R}=\{D(l)^{R}(1),D(l)^{R}(2),\cdots,D(l)^{R}(N)\} $$
    (16)
    $$ S_{G}=\{D(l)^{G}(1),D(l)^{G}(2),\cdots,D(l)^{G}(N)\} $$
    (17)
    $$ S=S_{R}+S_{G}=\{D(l)^{R}(1)+D(l)^{G}(1),\cdots,D(l)^{R}(N)+D(l)^{G}(N)\} $$
    (18)
  3. Step3.

    According to the order of S sequence, we pre-embed. Since overflow may occur during embedding, pre-embedding is performed before the formal embedding. Use Formula (19) to calculate the number of blocks pre-embedded \(N(N \geq \lceil \frac {len_{w}}{2} \rceil )\), lenw is the watermark sequence length, a is the linear increment coefficient.

    $$ N=a \times \lceil \frac{len_{w}}{2} \rceil \qquad \qquad a \geq 1 $$
    (19)
  4. Step4.

    Take the front N block in Ssort. Then take \(T^{\prime }=Max(D(l))\), \(T^{\prime }\) is the predicted shift threshold. Set the robust watermark embedding strength as G. Calculate the predicted offset \(B^{\prime }=T^{\prime }+G\).

  5. Step5.

    Use the \(B^{\prime }\) generated in step 4 to pre-embed. If all the watermark sequences can be embedded, then generate the local map (Fig. 7 shows the robust watermark local map of Lena) and let \(B=B^{\prime },T=T^{\prime }\). Otherwise, increase the value of a to increase the number of pre-embedded blocks and return to step 3. Figure 8 shows the flow chart of generating local map.

  6. Step6.

    According the local map generated in Step 5, perform three-level IWT on labeled small block of the R and G layer. Using the histogram shift method, as shown in Formula (2), embed the robust watermark in the low-frequency area.

  7. Step7.

    Perform inverse IWT on each embedded block, and replace the original image block to generate a watermarked image (\(\text {R}^{\prime }\), G’, B).

  8. Step8.

    Store the local map, the predicted shift threshold T and the robust watermark embedding strength G.

Fig. 6
figure 6

Heat map of differential value of R, G and B layers

Full size image
Fig. 7
figure 7

Robust watermark local map of Lena

Full size image
Fig. 8
figure 8

The flow chart of generating local map

Full size image

3.1.2 Robust watermark extraction

Extraction process of the robust watermark is as follows:

  1. Step1.

    Read the store T, G and the local map.

  2. Step2.

    Divide an image into blocks. According to the local map, perform IWT transform on the watermarked block. Use Formula (1) to calculate the difference \(D^{\prime }(l)\).

  3. Step3.

    This paper uses three watermark extraction schemes [15] to extract the watermark. The first two schemes use the control range method. The third scheme uses the K-means clustering algorithm, such as Formula (20–22).

    $$ w_{1}^{\prime}(i)=\left\{ \begin{array}{lr} 0 \qquad \qquad if \quad E^{\prime} \in [-T-\frac{G}{2},T+\frac{G}{2}]\\ 1 \qquad \qquad otherwise \end{array} \right. $$
    (20)
    $$ w_{2}^{\prime}(i)=\left\{ \begin{array}{lr} 0 \qquad \qquad if \quad E^{\prime} \in [-T-\frac{G}{3},T+\frac{G}{3}]\\ 1 \qquad \qquad otherwise \end{array} \right. $$
    (21)
    $$ w_{2}^{\prime}(i)=\left\{ \begin{array}{lr} 0 \qquad \qquad if \quad E^{\prime} \in class2\\ 1 \qquad \qquad if \quad E^{\prime} \in class1\ or \ class3 \end{array} \right. $$
    (22)
  4. Step4.

    The watermark sequence is extracted by Formula (20–22). The watermark is extracted from Formula (23).

    $$ w^{\prime}(i)=\left\{ \begin{array}{lr} w_{1}^{\prime}(i)\qquad if \quad w_{1}^{\prime}(i)=w_{2}^{\prime}(i),w_{1}^{\prime}(i)=w_{3}^{\prime}(i)\\ w_{1}^{\prime}(i)\qquad if \quad w_{1}^{\prime}(i)=w_{2}^{\prime}(i),w_{1}^{\prime}(i)\neq w_{3}^{\prime}(i)\\ w_{2}^{\prime}(i)\qquad if \quad w_{1}^{\prime}(i)\neq w_{2}^{\prime}(i),w_{2}^{\prime}(i)= w_{3}^{\prime}(i)\\ w_{3}^{\prime}(i)\qquad if \quad w_{1}^{\prime}(i)\neq w_{2}^{\prime}(i),w_{1}^{\prime}(i)= w_{3}^{\prime}(i) \end{array} \right. $$
    (23)

3.1.3 Robust watermark recovery

If the image has not been tampered with, restore the image with robust watermark. The recovery process is as follows:

  1. Step1.

    Use the local map to get the watermarked blocks in the R and G layers.

  2. Step2.

    The small blocks are transformed by IWT transformation to get the low-frequency areas. Then use Differential Histogram Shift to restore, as in Formula (5 and (6).

  3. Step3.

    After all the embedded blocks are recovered, perform inverse IWT to generate the restored image.

3.2 Tamper detection: fragile watermark scheme

3.2.1 Fragile watermark embedding

When embedding the fragile watermark, the predictive error length map is presented. The watermark can be embedded only when the position of the watermark sequence is 1. Please refer to Section 2.3 for more details. The specific process of fragile watermark embedding is as follows:

  1. Step1.

    The \(\text {R}^{\prime }\), \(\text {G}^{\prime }\) of the embedded watermark image is subjected to 8 × 8 blocks. Each block uses SHA-256 to generates a hash value and intercepts the front 18 bits as a fragile watermark.

  2. Step2.

    Layer B is divided into the same size blocks. Embed fragile watermarks in each block by Formula (8). Set the embedding length threshold μ, in the embedding process, if the length of the prediction error value is more than μ or the embedded pixels overflows, this position will not be embedded and the PELM is recorded as − 1. Otherwise, the PELM records the length of the prediction error value, as shown in Formula (24). An 8 × 8 block can be embedded with 18-bit watermark, so 18 prediction error lengths need to be stored. The PELM of a small block is shown in Fig. 9.

    $$ Map_{len}(x,y)=\left\{ \begin{array}{lr} -1 \qquad l\ \textgreater\ \mu\ or\ C^{\prime}\ \textgreater\ 255 \ or\ C^{\prime}\ \textless\ 0 \\ log_{2}(Pe)+1\qquad l\ \leq\ \mu\ and\ 0\ \leq\ C^{\prime}\ \leq\ 255 \end{array} \right. $$
    (24)
  3. Step3.

    After all the small blocks are embedded with fragile watermarks, the length map Maplen is stored.

Fig. 9
figure 9

Map of the predicted error length of a block

Full size image

3.2.2 Fragile watermark extraction

This paper uses a predictive error length map to embed fragile watermarks, which may cause some locations not to be embedded (Maplen(x,y) = − 1), and these locations will be skipped in the tamper detection. In this paper, fragile watermark extraction is composed of two stages. The first stage is the comparison of watermark hash sequences (step 1-3). The second stage is to restore the tamper detection map (step 4). The specific fragile watermark extraction process is as follows:

  1. Step1.

    Divide the watermarked image into 8 × 8 non-overlapping blocks. Then use \(\text {R}^{\prime }\) and \(\text {G}^{\prime }\) layers to generate hash sequences for each block H1(i),i ∈{1,2,...,4096}.

  2. Step2.

    Read the stored Maplen. Use Formula (12) to calculate the length of prediction error \(l^{\prime }\) for the B layer blocks. Then use Formula (13) to extract the watermark to generate H2(i),i ∈{1,2,...,4096}.

  3. Step3.

    The extracted watermark in Step 2 was compared with that generated by \(\text {R}^{\prime }\) and \(\text {G}^{\prime }\) layers, generate the initial tamper detection map Maptamper, as shown in Formula (25).

    $$ Map_{tamper}=\left\{ \begin{array}{lr} 0 \qquad \qquad H_{1}(i)=H_{2}(i)\\ 1 \qquad \qquad H_{1}(i)\neq H_{2}(i) \end{array} \right. $$
    (25)
  4. Step4.

    Set φ(0 ≤ φ ≤ 8 × 8), if at least φ positions in an n × n block are not embedded, mark this block as a suspected block. The possible locations of the suspected blocks are divided into three classes. Class1: center block, Class2: corner block, Class3: edge block, as shown in Fig. 10.

  5. Step5.

    Set a threshold 𝜃 to determine whether the suspected block, which is marked in Step 4, is tampered, . Different types of suspected blocks have different values of 𝜃k. k ∈{1,2,3} is the block class. In this paper, for the center block, k = 1,𝜃k = 5, for the corner block, k = 2,𝜃k = 1.5, for the edge block, k = 3,𝜃k = 2.9. Determine whether the suspected block is tempered by the neighboring 9 blocks. Use Formula (26) to update the position of the suspected block in Maptamper and obtain the final tamper detection map.

    $$ \left\{ \begin{array}{lr} T_{map}=0 \qquad \theta_{set}-\alpha \times Num_{miss} \geq \upbeta \times Num_{tramper} + \gamma \times Num_{suspect}\\ T_{map}=1 \qquad \theta_{set}-\alpha \times Num_{miss} < \upbeta \times Num_{tramper} + \gamma \times Num_{suspect} \end{array} \right. $$
    (26)
Fig. 10
figure 10

Suspected block classification

Full size image

3.2.3 Fragile watermark recovery

If the image is not tampered, restore the image with fragile watermark. The recovery process is as follows:

Step1. Formula (14) is used to calculate the restored prediction error value, and Formula (15) is used to obtain the restored center pixel Cr to restore the modified center pixel.

Step2. Restore all the modified center pixels to get the restored image.

4 Experimental results

The proposed scheme was implemented by Matlab R2021a in the PC with Intel Core i5-4200H CPU@2.80GHz and 16GB RAM, of which the OS is 64-bit Windows 10 professional. This paper uses Kodak and USC-ISUI data set for testing, the color image having size 512 × 512 × 3 and watermark image having size 32 × 32. The invisibility, reversibility, robustness and tamper detection of the proposed method has been evaluated in terms of Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) for watermarked image, Normalization cross correlation (NC) for watermark image, Accuracy (ACC) for tamper detection diagram. Section 4.1 describe the watermark invisibility and reversibility. Sections 4.2 and 4.3 represent the watermark robustness test and fragile test, respectively. Section 4.4 describe the watermark capacity.

4.1 Watermark invisibility and reversibility analysis

Figure 11 shows test images, and use the Formula (27) to balance invisibility and robustness. Change the target value by adjusting the parameters a1,a2,a3, and p. By changing the embedding strength G, the PSNR value, SSIM value and NC value will change, and the target value changes accordingly. The closer the target value is to 1, the better the balance. According to the target value, different images choose the best embedding strength G.

$$ target=a_{1} \times \frac{PSNR}{p}+a_{2} \times SSIM + a_{3} \times NC $$
(27)
$$ avgNC=\frac{{\sum}_{i=0}^{10}NC_{i}}{10} $$
(28)
Fig. 11
figure 11

28 images used in the experiment

Full size image

Table 1 shows the experimental parameters and results, where T and G are the predicted shift threshold and the robust watermark embedding strength when embedding different images, respectively. PSNR1 and SSIM1 are the values of the watermark and host image after the robust watermark is embedded, PSNR2 and SSIM2 are the values after the robust and fragile watermark are embedded, PSNR3 and SSIM3 are the values of the restored image and the host image, avgNC is the average of the watermark NC values after the watermarked image has been attacked by JPEG (QF = 40%), Histogram equalization, Gaussian noise (0.5%), Salt-pepper noise (0.5%), Speckle noise (1%), Average filter 3 × 3, Median filter 3 × 3, Motion filter 3 × 3, Gaussian LPF 3 × 3, and Crop 25%. The calculation formula of avgNC is shown in Formula (28). NCi is the NC value of the extracted watermark image and the original watermark image when carrying out different attacks. Table 1 shows the average PSNR2 is 43.234, and the SSIM2 values are all greater than 0.98, which indicates the good imperceptibility of the proposed scheme. The reversibility can be shown by PSNR3 and SSIM3, which is all INF and all 1, respectively. This shows that the image is completely reversible, that is to say, the watermarked image can be completely restored to the original image. The original image, watermarked image and the restored image of seven random selected in the test image are shown in Fig. 12.

Table 1 Experimental parameters and results
Full size table
Fig. 12
figure 12

The original image, watermarked image and the restored image of seven image random selected in the test image: (O1-O7) original image, (W1-W7) watermarked image, (R1-R7) restored image

Full size image

As can be seen from Table 1, the T values of Image 5, 8 and 20 are large, which means that the B value, the offset of the histogram, of these three images is also large, and is easy to cause pixel value overflow. Therefore, these three images cannot be used to embed watermarks and the PSNRs and SSIMs are not available. To explore the transformation of PSNR and NC with G. This paper uses a 32 × 32 watermark image and Lena, Baboon, Peppers, Airplane image to conduct experiments. Figure 13 shows the changes in PSNR and avgNC values of different images as G increases by 50 from 1. It can be seen that avgNC shows a logarithmic growth pattern and PSNR slowly declines. If G is approximately less than 10, avgNC increases sharply with G, and then slowly after that. It means that if G is kept in the range of about 10, the balance between robustness and invisibility reaches the best.

Fig. 13
figure 13

The changes of T, PSNR, avgNC value of different images as the G increases

Full size image

4.2 Watermark robustness test

In practice, the watermarked image may experience various distortions, such as JPEG attack, noise attack, filter attack, etc. Table 2 shows the comparison between the proposed scheme and [26]. NC and BER in the table are the average of all test images in Fig. 11. For Gaussian filter, Gaussian noise, Speckle noise and high QF JPEG compression attack, the proposed scheme is better than all the schemes in [26]. For Salt-pepper noise, Average filter and Median filter, The proposed scheme is better than or close to the scheme in [26]. It proves that this scheme has better robustness.

Table 2 Comparison with [26] NC and BER value. Bold represents the highest value
Full size table

Lossy compression technology is usually used to encode color images for efficient storage and communication. In this test, JPEG compression was used to attack watermark images. Figure 14 shows the comparison with the Su et al.’s scheme [23] after the JPEG attack. Figure 14 contains a comparison of four pictures of Lena, Baboon, Airplane, and Peppers. When QF> 0.5, the NC values of four pictures are above 0.8. When QF = 0.6,QF = 0.5,QF = 0.4, the proposed scheme has a higher NC value than [23] and is more robust.

Fig. 14
figure 14

The comparison with the Su et al.’s scheme [23] after the JPEG attack

Full size image

Noise attacks and filter attacks are classic attacks against digital watermarking. Watermarked images are easily and inevitably affected by noise during transmission. Moreover, filters of different sizes work on the watermarked image may also make the embedded watermark disappear. The test compares the proposed scheme with [4, 9, 23]. By adjusting the parameters of different algorithms, the imperceptibility of each algorithm is similar. In this case, the anti-noise attack and anti-filtering attack are tested. The comparison results are shown in Table 3. It can be seen that the NC value of the watermark extracted by the proposed scheme is better than [4, 9, 23] under Gaussian noise, Salt-pepper noise, Speckle noise, Average filter, Median filter, and Gaussian LPF attacks. The comparison with the schemes [4, 23] in terms of Gaussian noise and salt and pepper noise attacks is shown in Fig. 15. It can be seen from Fig. 15 that the NC values of the watermark extracted by the proposed scheme are all greater than [4, 23], the BER values are less than [4, 23]. This shows that the scheme proposed in this paper can effectively resist noise attacks and is better than [4, 23].

Table 3 The NC values of noise attack test for Lena. Bold represents the highest value
Full size table
Fig. 15
figure 15

The comparison with the Su et al.’s scheme [23] and Duan et al.’s method [4] after the Noise Attack

Full size image

4.3 Watermark fragility test

Watermarked images may be subjected to malicious attacks In this paper, the PEE method is improved, and the PELM is proposed to reduce the impact on the invisibility of the watermark. Table 4 shows comparison of imperceptibility between PELM method and PEE. The PELM method proposed in this paper is superior to the PEE in both PSNR value and SSIM value.

Table 4 Comparison of imperceptibility between PELM and PEE
Full size table

Common malicious attacks mainly include random block attacks and object attacks. Random block attack is an attack on blocks of random size and location. Object attack adds or deletes a specified area of an image. This paper proposes a tampering detection map correction program, and in the test, select α = 1,β = 0.4 as the value in Formula (27) for experiment. Figure 16 shows Lena being attacked by random blocks. Figure 17 shows the results of object attacks. The comparison of tamper detection accuracy between the proposed scheme and the schemes of Duan et al. [4] and Kamili et al. [9] is shown in Fig. 18. When compared with [4, 9], except for the S&P random block attack, the ACC is slightly lower than other schemes, and the rest of the random block attack and object attack are better than the other two schemes.

Fig. 16
figure 16

Random block attack and tamper detection: (T1-T6) Images after random block attack: (T1) blurring, (T2) salt and pepper noise, (T3) Gaussian noise, (T4) average filter, (T5) sharpening, (T6) cropping. A1-A6 are tamper detection diagrams extracted from the scheme, B1-B6 are tamper detection diagrams extracted from Duan et al.’s scheme [4], C1-C6 are tamper detection diagrams extracted from Kamili et al.’s scheme [9]

Full size image
Fig. 17
figure 17

Results of object attacks: (W1-W6) watermarked image (T1-T6) images after object attack: (T1) attacked image in “Peppers” by adding a new pepper, (T2) attacked image in “Airplane” by adding another airplane, (T3) attacked image in “Baboon” by adding an eye, (T4) attacked image in “Sail-boat” by removing the boat, (T5) attacked image in “House” by removing a window, and (T6) attacked image in “Airplane” by removing the “USA AIR FORCE”, and (A1-A6) tamper detection diagrams extracted from the proposed scheme, (B1-B6) tamper detection diagrams extracted from Duan et al.’s scheme [11], (C1-C6) tamper detection diagrams extracted from Kamili et al.’s scheme [9]

Full size image
Fig. 18
figure 18

The comparison with the Duan et al.‘s scheme [4] and Kamili et al.‘s scheme [9] after tamper detection

Full size image

4.4 Watermark capacity

This paper analyzes the robust watermark capacity by embedding rate. The bits per pixel (bpp) is used to represent the pure payload (that is, the watermark capacity that can be embedded per pixel). The watermark in this paper is a 32 × 32 binary image, and the host image is a 512 × 512 color image, thus, the embedding capacity is (32 × 32)/(512 × 512 × 3) = 0.0013bpp. The capacity of the method [4, 23] is the same as that of the proposed in this paper. The capacity of the method [9] is (64 × 64)/(512 × 512 × 3) = 0.0052bpp. Obviously, the proposed method has the lowest capacity. This is because in order to achieve better robustness and invisibility, only one bit of watermark is embedded in a 16 × 16 image block. Therefore, in the follow-up work, we will consider how to embed more watermarks while maintaining robustness and invisibility.

5 Conclusion

This paper introduces a reversible multi-watermarking scheme for color images with robustness and fragility. The main work is as follows: 1) IWT and histogram shift are used to realize robust watermarking; 2) The pre-embedding method can select a more appropriate embedding location. In addition, the PEE algorithm is optimized by PELM to reduce the embedding capacity. These optimizations greatly improve the invisibility of the watermark; 3) a tamper detection map correction scheme is proposed to improve the accuracy of tamper detection.

The experimental results show that the average PSNR value is 43.234 dB, which has good imperceptibility. Meanwhile, it is robust to common attacks. Especially in noise attacks, it achieves the highest robustness compared to those proposed in literature [4, 9, 22]. When the watermarked Lena is attacked by 1% pepper-and-salt noise, the NC value is higher than 0.98. The accuracy of tamper detection reaches 99.9% under object attacks, which exceeds the schemes proposed by Kamili et al. [9] and Duan et al. [4].

However, there is still much room for improvement in our scheme. 1) After embedding the watermark, some pixels may overflow and cannot be further embedded. 2) The location map and PELM in this paper are lack of security considerations during storage and transmission. If they are attacked, the watermark cannot be extracted. Therefore, in the future work, this problem will be solved by cryptography or hiding them into the image. 3) Pre-embedding consumes a lot of time. How to reduce the time overhead is also a work need to do in the future.