1 Introduction and contributions

With the development of digital image industry, people can utilize benefits of digital images which help them modify a photo easily and feasibly. However, some image forgers maliciously falsify digital images. Thus, it is urgent to propose a digital forensic tool against malicious image falsification and restore the trustworthiness to digital images. Digital image forensics can be classified into two categories. One is defined as active forensics, referring to as digital watermarking. Since the embedding mechanism has to be available and the credibility of information embedded in the image remains questionable, the widespread utilization of digital watermarking has been substantially limited. The other is defined as passive forensics. Different from digital image watermarking, widely-adopted passive forensics authenticates digital images without any prior information on which forensic investigators mainly focus(see [8, 27, 31, 32]).

Many methodologies have been proposed for detecting image forgery. Prior to discussion of image forensics, it should be noted that in this context, we do not categorize steganalysis including LSB replacement and matching (see [35]) into the field of image forgery detection. Using sensor pattern noise as camera signature, it was proposed to identify the origin of the acquired image (see [3, 18]). Based on the physical noise model proposed in [33], the designed detector first identified source camera model by using RAW format images. Moreover, inspired by this pioneer work, the authors of [23] proposed the improved noise model for identifying individual source camera device. By investigating the artifacts of the Color Filter Array (CFA) pattern, authors of [12, 22] distinguished a photographic image (PIM) from a computer generated (CG) one. In the practical operation of image forgery, a manipulated image is probably compressed twice. Thus, JPEG double-compressed detector is also an effective auxiliary tool to authenticate tampered images (see [7]). To distort the truth, a real object in the digital photo is possibly concealed by forgers. Thus, it is necessary to propose a forensic tool to detect copy-move or splicing manipulation (see [1, 4, 11, 17]). Besides, image hashing technique can also help forensic investigators authenticate manipulated images (see [28]).

1.1 State of the art

To create convincing forged images, a forger needs to use some geometric operations which require a resampling of pixels. Resampling is a post-camera processing technique. In general, it is proposed not to classify resampling among the image tempering technique. However, resampling of pixels is the basis of many forgeries because it is involved in almost all geometric modification (rotation, resizing, etc. ...). While the evidence of resampling operation within an image does not necessarily imply that the image has been tempered it is a very important clue. Thus, resampling detectors play a very important role as an auxiliary forensic tool. Additionally, resampling falsification often happens together with other forgery techniques, for instance, copy-move or splicing forgery. For instance, in order to enhance the visual realism, splicing and copy-move forgeries often require scaling and/or rotation operations which requires resampling operation.

To distinguish a resampled and non-resampled image is the subfield of digital image forensics on which this paper focuses. Forensic methods for exploiting the artifacts of resampled images can generally be divided into two following fundamental categories:

  1. 1.

    Methods in the first category focus on identifying the linear correlation of resampled images in the spatial domain. Authors of [21] first proposed a linear predictor to expose the correlation existing among neighboring pixels of the resampled images by using Expectation Maximization (EM) algorithm (see [6]). Although EM algorithm was capable of estimating the linear interpolated parameters, iterative computation costs a plenty of time. Driven by this pioneer work, it was proposed to analyze the relevance between the EM detector and the second-derivative one, and design an equivalent accelerated and simplified detector in [14]. However, in place of estimating the interpolation parameters, the fixed parametric predictor indeed improves the efficiency of detection, yet the detection accuracy can not be guaranteed. The problem of how to strike the balance between the computation efficiency and estimation accuracy remains open. Instead of using a single predictor, authors of [15] proposed to use a global predictor to detect a resized image. Its limitation is that the proposed algorithm can only be used for detecting resized images excluding other geometric manipulation such rotation or shearing. In addition, these methods can detect the linear transformation, yet can not estimate the interpolation period.

  2. 2.

    Methods in the second category investigate the second or high-order derivative signal of resampled images in the frequency domain. By exploiting the second-derivative algorithm, it was proposed to uncover the periodicity of an interpolated image in [13]. This method can estimate the interpolation period, yet only applied in the resized images. By analyzing the variance of the n-th order derivative, it was proposed to expose the trace of a resampled image in [19]. However, the linear transformation matrix can not be approximated. Inspired by the algorithm of [13], it was proposed to analyze the relationship between the rotation angle and the frequency in the spectrum of the image’s edge map (see [34]). The limitation of rotation forgery detection is that this method could not be comprehensively used. By using supervised learning method (such as Support Vector Machine, SVM), it was proposed to detect whether the inspected image was resampled (see [9, 10]). Although those methods could achieve high detection accuracy, supervised statistical learning was time-consuming. In addition, several problems such as the robustness to training and testing set mismatch remain open. For approximating the linear transformation of an interpolated image, authors of [29] proposed to analyze the second-derivative signals of the image in both the row and column directions. Although most of these methods could estimate the linear transformation matrix and interpolation period, they did not establish the detection performance statistically.

1.2 Contributions of the paper

First, the proposed approach detects a one-dimensional (1-D) resampled signal and estimates the interpolated period. Second, by uncovering the linear correlation of pixels existing in an image in the spatial domain, the proposed algorithm authenticates a two-dimensional (2-D) resampled image. Third, by designing a practical Likelihood Ratio Test (LRT), the proposed algorithm detects a batch of resampled images, and warrants a prescribed False Alarm Rate (FAR), and achieves good detection performance. Hence, the main contributions are the followings:

  • To deal with 1-D resampled signal, it is first proposed to extract the residual noise of the inspected signal using a linear parametric model. Meanwhile, we design the detector based on the probability of residual noise (or prediction errors).

  • By transforming the probability of the residual noise into Fourier spectrum, it is proposed to establish an estimator for estimating the resampling factor.

  • Based on another linear parametric model, the probability of residual noise extracted from the inspected image is estimated. Then, we expose the periodic artifact of the resampled image which can be described as symmetrical bright spots scattering in Fourier domain.

  • To deal with a large scale of resampled images, based on the estimated probability of residual noise, we propose to design a practical LRT and detect resampled compressed or uncompressed images.

In this context, it should be noted that we can provide a new approach of detecting resampled images based on our proposed statistical models. However, we are incapable of designing an most optimal power test with known statistical properties, which is the limitation of the proposed algorithm.

1.3 Organization of the paper

This paper is organized as follows. Section 2 first recalls the principle of 1-D signal resampling. Then, the linear parametric model is proposed for dealing with the nuisance parameters. Meanwhile, the 1-D resampling detector is introduced. In Section 3, the linear correlation of a 2-D resampled image is first illustrated. At the same time, it is proposed to design a test of detecting a single resampled image. Next, Section 4 presents a practical LRT for detecting a batch of resampled images. Finally, Section 5 presents numerical results of the proposed detector on real images. Section 6 concludes this paper.

2 One-dimensional signal resampling and its artifact

2.1 Problem statement of one-dimensional signal resampling

In general, resampling can be proceeded by three steps: up-sampling; interpolation; down-sampling (see details in [20]). Since linear interpolation is widely-adopted, for simplicity and clarity, it is assumed that the interpolation algorithm studied in the present paper is linear. In the following paragraph, a simple illustration of the resampling process is displayed (see Fig. 1).

Fig. 1
figure 1

Illustration of resampling 1-D signal by a factor \(\frac {p}{q}=\frac {4}{3}\): (a) 1-D original signal, (b) up-sampled signal by a factor p=4, (c) interpolated signal, (d) resampled signal after down-sampling by using a factor q=3

Full size image

Without loss of generality, a 1-D discrete signal is created (see Fig. 1a), denoted the vector x={x[t]} t∈{1,...,T}, where T denotes the length of the vector. First, by up-sampling with the factor p, a new up-sampled vector x u ={x u [t]} is generated with the length p(T−1)+1 (see Fig. 1b). In the case of t=1,2..,T, x u [p(t−1)+1] = x[t] ; otherwise, x u [t]=0. Second, by convolving the vector x u with a linear low-pass filter h[t], the interpolated vector x i ={x i [t]}, t∈{1,...,p(T−1)+1} is obtained, where x i [t] = x u [t]∗h[t] (see Fig. 1c). Third, by down-sampling with the factor q, a new down-sampled vector x d ={x d [t]} with the length \(\lfloor \frac {p}{q}(T-1)+1\rfloor \), where x d [t] = x i [q(t−1)+1], \(t=1,2,...,\lfloor \frac {p}{q}(T-1)+1\rfloor \) (see Fig. 1d). Let us denote y={y[t]} resampled signal where y≡x d . It should be noted that \(p,q\in \mathbb {N}^{+}\). Then, it is immediately proposed that the resampled signal y is described by the following linear equation:

$$ \textbf{y} = \mathbf{B}_{\frac{p}{q}}\textbf{x} $$
(1)

where y={y m }, m∈{1,...,M}, denoted a resampled signal, x={x n }, n∈{1,...,N}, denoted an original signal. A M×N linear resampling matrix \(\mathbf {B}_{\frac {p}{q}}\) controls the resampling procedure, including up-sampling, interpolation, and down-sampling.For instance, the linear resampling matrix \(\mathbf {B}_{\frac {4}{3}}\) (see Fig. 1) takes the form:

$$\mathbf{B}_{\frac{4}{3}} = \left( \begin{array}{cccccccl} 1 & 0 & 0 & 0 & {\ldots} \\ 0.25 & 0.75 & 0 & 0 & {\ldots} \\ 0 & 0.5 & 0.5 & 0 & {\ldots} \\ 0 & 0 & 0.75 & 0.25 & {\ldots} \\ 0 & 0 & 0 & 1 & {\ldots} \\ {\vdots} & {\vdots} & {\vdots} & {\vdots} & {\ddots} \\ \end{array}\right) $$

where \(\mathbf {B}_{\frac {4}{3}}\) has the period 4. Since the linear resampling matrix \(\mathbf {B}_{\frac {p}{q}}\) introduces the periodicity, the resampled signal y owns the period p. Moreover, the expectation of the periodic signal y m should be linear combination from the neighbouring samples, which is defined by:

$$ \hat{y}_{m} = \sum\limits_{k=-L}^{L}\alpha_{k} y_{m+k} $$
(2)

where α k denotes the weighted factor with α 0=0 and 2L (\(L\in \mathbb {N}^{+}\)) the number of used neighbouring samples. In the present paper the moments of (2) is not in detail, the interested reader is referred to [21]. It should be noted that the linear combination (2) can not hold true in each sample along the signal y m , but only be defined in the interval of p. In the practical resampling, if an up-sampling factor p set as 1 while a down-sampling factor q>1, the resampled signal does not expose the periodicity, and then its expectation can not be formulated by linear combination (2).

It is proposed to use a linear parametric model to deal with a 1-D resampled signal y. To this end, the vector \(\textbf {y}=\{y_{1},\ldots ,y_{M}\}^{T}\) is defined by:

$$ \textbf{y} \sim \mathcal{N}(\boldsymbol{\mu} , \sigma^{2} \mathbf{I}_{M}) =\boldsymbol{\mu}+\boldsymbol\epsilon $$
(3)

where μ = (μ 1,…,μ M )T denotes the expectation of y and 𝜖 = (𝜖 1,…,𝜖 M )T denotes residual noise following the Gaussian model with the variance \(\sigma ^{2}\mathbf {I}_{M}\) and I M the identity matrix of size M×M. Furthermore, μ can be described with the following linear parametric model:

$$ \boldsymbol{\mu} = \mathbf{A} \boldsymbol{\omega} $$
(4)

where A is a known full rank matrix of size M×N, with M>N, and nuisance parameters ω is a N×1 vector of describing the expectation of y. In the present paper, the matrix A can be described as:

$$\mathbf{A} = \left( \begin{array}{cccccc} a_{1} & {\ldots} & a_{L} & a_{L+2} &{\ldots} & a_{2L+1}\\ a_{2} & {\ldots} & a_{L+1} & a_{L+3} &{\ldots} & a_{2L+2}\\ {\vdots} & {\ddots} & {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ a_{m} & {\ldots} & a_{L+m-1} & a_{L+m+1} &{\ldots} & a_{2L+m}\\ {\vdots} & {\ddots} & {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ a_{M-2L} & {\ldots} & a_{M-L-1} & a_{M-L+1} &{\ldots} & a_{M}\\ \end{array} \right), $$

where m denotes an index of each column vector and 2L a number of used neighbouring samples, with M>3L. It should be noted that the expectations of the first and last L samples are not considered in our proposed model. In practice, y denotes an observation of a.

The idea of using such a linear parametric model [5, 22, 36] is that it allows an easy elimination of nuisance parameters. To apply this theory, let us define \(C\left (\mathbf {A}\right ) \subseteq \mathbb {R}^{M}\) the column space spanned by A, with dim \(\left (C\left (\mathbf {A}\right )\right )\) = rank (A) = N and \(C\left (\mathbf {A}\right )^{\bot } \subseteq \mathbb {R}^{M-N}\) its orthogonal complement, sometimes referred to as the “parity space”, with dim \((C\left (\mathbf {A}\right )^{\bot })= M-N\). The projection of observation vector y onto the parity space \(C\left (\mathbf {A}\right )^{\bot }\) is obtained by n = W y where the matrix W verifies, among others, the following useful properties:

$$ \mathbf{W A} = \mathbf{0} \; \text{ and } \; \mathbf{W}\mathbf{W}^{T} =\mathbf{I}_{M-N} . $$
(5)

Hence, the projection of observation vector y onto the parity space C(A)⊥ yields: \(\mathbf {n} = \mathbf {W} \textbf {y} = \mathbf {W} \mathbf {\epsilon } \sim \mathcal {N}(0,\sigma ^{2}\mathbf {I}_{M-N})\). Note that the use of projection matrix W can be replaced by using the Maximum Likelihood Estimation (MLE). By rejecting the nuisance parameter, let us define the estimation of n as

$$ \widehat{\mathbf{n}}=\textbf{y} - \mathbf{A} (\mathbf{A}^{T} \mathbf{A})^{-1}\mathbf{A}^{T}\textbf{y} . $$
(6)

Since the resampled samples are interpolated from their neighbourhood non-resampled samples, the residual noise obtained from the difference between the resampled samples and their estimation approximately follows the Gaussian distribution with zero expectation. To verify the effectiveness of our proposed model, the simulated experiment is illustrated. A set of random variables of 1000 samples, referring to as the original signal, is generated. Then, by using a resampling factor (RF) \(\frac {p}{q}=\frac {2}{1}\), the resampled signal is obtained (see Fig. 2a). Based on our proposed linear parametric model, the residual noise of resampled signal is acquired, which can be fitted by using the Gaussian distribution (see Fig. 2b). Similarly, it is also proposed to give the empirical results from the residual noise of original signal and its Gaussian model fitting (see Fig. 2c). Obviously, the resampled residual noise fits the Gaussian model better, which verifies our proposed assumption that residual noise of resampled signal approximately follows the Gaussian distribution. By contrast, it hardly holds true that the residual noise of original signal follows the Gaussian distribution.Footnote 1

Fig. 2
figure 2

Illustration of resampled signal with RF = 2 and statistical distributions of the residual noise (resampled and original version)

Full size image

2.2 Exposing one-dimensional signal resampling

Based on the linear parametric model, the extracted residual noise denoted n={n i } i∈{1,...,I} follows the Gaussian distribution (see Fig. 2b). For simplifying the calculation, it is assumed that the residual noise from non-resampled/original signal is uniformly distributed (see [21]). Thus, the problem of detecting between the original and resampled signal can be formulated by the following two hypotheses:

$$ \left\{\begin{array}{ll} \mathcal{H}_{0} \,=\, \{n_{i} \!\sim \mathcal{U}(n_{min},n_{max})) , \forall i\,=\,(1,...,I) \},\\ \mathcal{H}_{1} \,=\, \{n_{i} \!\sim \mathcal{N}(0,\sigma^{2}) , \forall i\,=\,(1,...,I) \}, \end{array}\right. $$
(7)

where under hypothesis \(\mathcal {H}_{0}\), the residual noise of the inspected signal follows the Uniform distribution; under hypothesis \(\mathcal {H}_{1}\), the residual noise is normally distributed with zero expectation and σ 2 variance. It is noted that n m i n denotes the minimum value of the noise and n m a x the maximum value, assuming that n m i n ≠n m a x .

Then, by using Bayes’ rule, let us immediately denote the conditional probability of each noise sample from the resampled signal by:

$$ \Pr\{n_{i}\in \mathcal{H}_{1}\mid n_{i}\} = \frac{\Pr\{n_{i}\mid n_{i}\in \mathcal{H}_{1}\}\Pr\{n_{i}\in \mathcal{H}_{1}\}}{\sum\nolimits_{k=0}^{1}\Pr\{n_{i}\mid n_{i}\in \mathcal{H}_{k}\}\Pr\{n_{i}\in \mathcal{H}_{k}\}}, $$
(8)

where it is assumed that the priors \(\Pr \{n_{i}\in \mathcal {H}_{0}\}\) and \(\Pr \{n_{i}\in \mathcal {H}_{1}\}\) are both equal to \(\frac {1}{2}\). Then (7) is rewritten as follows:

$$ \left\{\begin{array}{ll} \Pr\{n_{i}\mid n_{i}\in \mathcal{H}_{0}\} = \frac{1}{n_{max}-n_{min}}\\ \Pr\{n_{i}\mid n_{i}\in \mathcal{H}_{1}\} = \frac{1}{\sqrt{2\pi\sigma^{2}}}\exp\left( \frac{-{n_{i}^{2}}}{2\sigma^{2}}\right). \end{array}\right. $$
(9)

where in the practical detection, the variance σ 2 can be replaced by its estimate \(\widehat {\sigma }^{2}\) by using MLE as follows:

$$ \widehat{\sigma}^{2} = \frac{1}{I-1}\sum\limits^{I}_{i=1}(n_{i} - \frac{1}{I}\sum\limits^{I}_{i=1}n_{i})^{2}, $$
(10)

where I denotes the total number of the pixels.

It is proposed to use the frequential representation of the probability of resampled residuals, denoted N and formally defined as follows:

$$ \mathbf{N} \,=\,\lvert \textbf{h} \ast \lvert f[\Pr\{\mathbf{n}\in \mathcal{H}_{1}\mid \mathbf{n}\}] \rvert \ \rvert, $$
(11)

where a vector h=[1/4,1/4,−1,1/4,1/4] denotes a 1-D high-pass filter, f[⋅] represents the calculation of 1-D Discrete Fourier Transform (DFT). It should be noted that the high-pass filter aims at removing the lower frequential components for better visually displaying. Due to the periodical artifact caused by resampling operation, the spike-like peak value appears in the frequency domain which is the most distinctive artifact of the resampled signal (see Fig. 3b and c).

Fig. 3
figure 3

N (frequential representation of the probability) comparison of residual noise between original and resampled signal in Fourier domain, (a) original signal, (b) resampled signal by a factor \(\frac {p}{q}=\frac {4}{1}\), (c) resampled signal by a factor \(\frac {p}{q}=\frac {4}{3}\), (d) resampled signal by a factor \(\frac {p}{q}=\frac {1}{2}\)

Full size image

To verify the effectiveness of our proposed algorithm, it is proposed to compare the original signal with the resampled one by using different RFs. As Fig. 3 illustrates, the resampled signal (see Fig. 3b and c) exposes the obvious peaks in the frequency domain, but the original signal (see Fig. 3a) has nothing. Moreover, surprisingly, the resampled period is accurately estimated by:

$$ \hat T = \frac{1}{f_{p}} $$
(12)

where \(\hat T\) denotes the estimated period or interpolated rate of the resampled signal, which equals to the up-sampled factor p. f p is the normalized frequency, corresponding to the position of the first peak in Fourier domain. By counting the number of the local peaks with a threshold τ 1, we can design a test of detecting a resampled signal.

It should be noted that our proposed algorithm can not differentiate between two resampled signals by using the same up-sampled factor. For instance, the peaks associated with a resampled factor \(\frac {4}{1}\) (see Fig. 3b) appears similar to those by a resampled factor \(\frac {4}{3}\) (see Fig. 3c). In addition, in the case of the resampled signal by an up-sampled factor p=1, where it has no obvious peaks (see Fig. 3d), it can not be distinguished with the original signal. In this case, each sample of the resampled signal exposes no periodic linear correlation with its neighbouring ones. Thus, our proposed algorithm is invalid.

3 Two-dimensional signal resampling and its artifact

In this section, inspired by extracting the artifact of the 1-D resampled signal (11), we can extend the algorithm from a 1-D signal to a 2-D image straightforward. Compared with a 1-D resampled signal, a 2-D resampled image has more complex linear correlation. A 1-D signal is interpolated by one direction, but a 2-D image is interpolated by the horizontal and vertical direction simultaneously. When a 2-D image is resampled only by the horizontal or vertical direction, it will cause the image visual distortion, which is not discussed in this context.

3.1 Problem statement of two-dimensional signal resampling

Similarly, a procedure of resampling a 2-D image has three steps: up-sampling by the horizontal and vertical direction; interpolation by two directions; down-sampling by two directions. Then similar to the linear combination of 1-D signal, each pixel intensity of the resampled 2-D image correlates to its surrounding ones. Next, let us denote an original grey-level image as X={x i,j } i∈{1,...,I}, j∈{1,...,J}, where I and J denote the height and the width of the matrix X. The case of color image with three channels can be obtained straightforward.

Let us denote the linear correlation between each pixel sample and its neighbouring ones by the following:

$$ x_{i,j} = \sum\limits_{p=-L}^{L}\sum\limits_{q=-L}^{L}\alpha_{p,q}x_{i+p,j+q} $$
(13)

where α p,q denotes a weighted factor with the case of α 0,0=0, (2L+1)2−1 (\(L\in \mathbb {N}^{+}\)) the number of used neighbouring samples, x i,j i∈{2,...,I−1}, j∈{2,...,J−1} denotes a pixel intensity. Enlarging L the range of neighbouring indeed improves the accuracy of describing the linear correlation, but at the cost of high computation. For simplicity, L=1 is a good tradeoff between complexity and accuracy. Then let us rewrite a resampled image X={x j }, the vector \(\textbf {x}_{j}=\{x_{2,j},...,x_{I-1,j}\}^{T}\), where j∈{2,...,J−1}. Immediately, it is proposed to use a linear parametric model to deal with the 2-D image. To this end, the vector x j is defined:

$$ \textbf{x}_{j} \sim \mathcal{N}(\boldsymbol{\mu}_{j} , {\sigma^{2}_{j}} \mathbf{I}_{I-2}) =\boldsymbol{\mu}_{j}+\mathbf{n}_{j} $$
(14)

where μ j =(μ 2,…,μ I−1)T denotes the expectation, each Gaussian-distributed column vector n j =(n 2,…,n I−1)T takes variance \({\sigma ^{2}_{j}}\mathbf {I}_{I-2}\) and I I−2 the identity matrix of size (I−2)×(I−2). Furthermore, μ j can be described with the following linear parametric model:

$$ \boldsymbol{\mu}_{j} = \mathbf{A}_{j} \boldsymbol{\alpha}_{j} $$
(15)

where A j is a known full rank matrix of size (I−2)×8, with I>10, and α j ={α p,q } where index (p,q)∈{−1,0,1} excluding (0,0) is a 8×1 vector describing the expectation of x j . Then, similar to (6), let us define the estimation of n j as

$$ \hat{\mathbf{n}}_{j}=\textbf{x}_{j} - \hat{\textbf{x}}_{j} = \textbf{x}_{j} - (\mathbf{A}_{j} ({\mathbf{A}_{j}^{T}} \mathbf{A}_{j})^{-1}{\mathbf{A}_{j}^{T}})\textbf{x}_{j} . $$
(16)

Note that in the present paper, the chosen linear parametric model dealing with the 2-D image is the following matrix A j :

$$ \mathbf{A}_{j} = \left[\textbf{x}_{j}^{\!(\!-1\!,\!-1\!)\!}, \textbf{x}_{j}^{\!(\!-1\!,\!0\!)\!}, \textbf{x}_{j}^{\!(\!-1\!,\!1\!)\!}, \textbf{x}_{j}^{\!(\!0\!,\!-1\!)\!}, \textbf{x}_{j}^{\!(\!0\!,\!1\!)\!}, \textbf{x}_{j}^{\!(\!1\!,\!-1\!)\!}, \textbf{x}_{j}^{\!(\!1\!,\!0\!)\!}, \textbf{x}_{j}^{\!(\!1\!,\!1\!)}\right] $$
(17)

where a matrix \(\mathbf {X}^{(p,q)} \,=\, \{\textbf {x}_{j}^{(p,q)}\}\), each column vector \(\textbf {x}_{j}^{(p,q)}\,=\,\{x_{2+p,j+q},...,x_{I+p-1,j+q}\}^{T}\), where j∈{2,...,J−1} and index (p,q)∈{−1,0,1} not including (0,0).

3.2 Exposing two-dimensional signal resampling

By using the linear parametric model, the residual noise can be extracted from an inspected image. Let us denote N={n 2,...,n J−1} residual noise in which each residual column vector n j ={n i,j }, i∈{2,...,I−1}. Thus, the problem of detecting between the non-resampled and resampled image can be formulated by the following two hypotheses:

$$ \left\{\begin{array}{ll} \mathcal{H}_{0} \,=\, \{ n_{i,j} \!\sim \mathcal{U}(-255,255)) , \forall i\,=\,(2,...,I-1) \},\\ \mathcal{H}_{1} \,=\, \{ n_{i,j} \!\sim \mathcal{N}(0,{\sigma_{j}^{2}}) , \forall i\,=\,(2,...,I-1) \}, \end{array}\right. $$
(18)

where j∈{2,...,J−1}, under hypothesis \(\mathcal {H}_{0}\), the residual noise of the non-resampled image follows the Uniform distribution;Footnote 2 under hypothesis \(\mathcal {H}_{1}\), the residual noise of the resampled image is normally distributed with zero mean and \({\sigma _{j}^{2}}\) variance. Besides, it should be noted that due to the pixel intensity of a gray-level image ranging from 0 to 255, residual noise (difference between the pixel’s intensity and its expectation value) falls in the range from -255 to 255. Then, by using Bayes’ rule, let us immediately denote the conditional probability of each residual sample:

$$ \Pr\{n_{i,j}\!\in\! \mathcal{H}_{1}\!\mid\! n_{i,j}\}\,=\,\frac{\Pr\{n_{i,j}\!\mid\! n_{i,j}\!\in \!\mathcal{H}_{1}\}\Pr\{n_{i,j}\!\in\! \mathcal{H}_{1}\}}{\sum\nolimits_{k=0}^{1}\!\Pr\!\{n_{i,j}\!\mid \!n_{i,j}\!\in\! \mathcal{H}_{k}\}\!\Pr\!\{n_{i,j}\!\!\in\!\!\mathcal{H}_{k}\}} , $$
(19)

where it is assumed that the priors \(\Pr \{n_{i,j}\in \mathcal {H}_{0}\}\) and \(\Pr \{n_{i,j}\in \mathcal {H}_{1}\}\) are both equal to \(\frac {1}{2}\). Then (18) is rewritten as follows:

$$ \left\{\begin{array}{ll} \Pr\{n_{i,j}\mid n_{i,j}\in \mathcal{H}_{0}\} = \frac{1}{510},\\ \Pr\{n_{i,j}\mid n_{i,j}\in \mathcal{H}_{1}\} = \frac{1}{\sqrt{2{\pi\widehat\sigma_{j}^{2}}}}\exp\left( \frac{-n_{i,j}^{2}}{2{\widehat\sigma_{j}^{2}}}\right), \end{array}\right. $$
(20)

where the variance \({\widehat \sigma _{j}^{2}}\) can be estimated by using MLE as follows:

$$ \widehat {\sigma_{j}^{2}} = \frac{1}{I-3}\sum\limits^{I-1}_{i=2}(n_{i,j} - \frac{1}{I-2}\sum\limits^{I-1}_{i=2}n_{i,j})^{2}, $$
(21)

where I denotes the total number of pixels in each column j.

Then it is proposed to use the frequential representation of the probability of residual noise, denoted N 2D and formally defined as follows:

$$ \mathbf{N}^{2D} \,=\,\lvert \mathbf{H} \ast \lvert \mathcal{F}[\mathbf{P}_{map}] \rvert \, \rvert, $$
(22)

where H denotes a 2-D high-pass filter, \(\mathcal {F}[\cdot ]\) represents the calculation of 2-D DFT. \(\mathbf {P}_{map}=\{\Pr \{n_{i,j}\!\in \! \mathcal {H}_{1}\!\mid \! n_{i,j}\}\}\) , i∈{2,...,I−1}, j∈{2,...,J−1} denotes the so-called probability map.

A 2-D resampled image exposes bright spots surrounding the central spot (similar to peaks in the 1-D signal) distributed symmetrically in Fourier domain, which is the critical resampling artifact of a resampled image. Due to the fact that the interpolated pixel obtained from its neighbourhood pixels, the periodic characteristic is uncovered unavoidably in the frequency domain. Therefore, the symmetric bright spots appear in Fourier domain when an image is resampled. Based on the assumed distributions (18), together with hypothesis theory, we can establish a practical LRT of detecting a resampled image. The specific discussion will be extended in the following section.

4 Design of the practical LRT for resampling detection

Prior to our proposed algorithm in this paper, some state-of-the-art methods confine their ideas to detection of the peak values (bright spots) in Fourier domain. Namely, by counting the number of bright spots, the designed test detects if the inspected image is resampled. Therefore, that proposed test has to estimate the suitable threshold for selecting bright spots existing in the Fourier domain (see [14, 21]). However, since some post-cameraFootnote 3 operations such as compression possibly generate some bright spots which are not caused by image resampling. That will to some extent impact the detection accuracy of that prior-art test.

In this section, it is proposed to solve the problem of resampling detection by designing a practical LRT, which has been verified to be a very effective algorithm in [23–26]. A flowchart of the proposed algorithm is illustrated in Fig. 4. By using a linear parametric model, residual noise can be extracted from the inspected image. Based on the statistical distributions of residual noise under two hypotheses, it is proposed to design a practical LRT. In this practical LRT, we first estimate the unknown parameter, and meanwhile calculate the LR, and finally detect the inspected image based on the given threshold.

Fig. 4
figure 4

A flowchart of the proposed algorithm

Full size image

For solving the statistical detection problem (18), it follows from the Neyman-Pearson lemma [16](@var1@Theorem 3.2.1@var1@) that the LRT is optimal in the sense described below. For definition, let

$$ \mathcal{K}_{\alpha}=\left\{ \delta: \sup_{\sigma_{j}} \mathbb{P}_{\mathcal{H}_{0}}[\delta(\mathbf{N})=\mathcal{H}_{1}]\leq\alpha\right\} $$
(23)

be the class of tests, solving problem (18), with an upper-bounded FAR α. Here \(\mathbb {P}_{\mathcal {H}_{j}}[\cdot ]\) is the probability under \(\mathcal {H}_{j},j\in \{0,1\}\), and the supremum over model parameter σ j can be understood as the distribution parameter ensuring that the FAR α can not be exceeded. Among all tests in \(\mathcal {K}_{\alpha }\) the LRT is the most powerful test, it maximizes the detection power:

$$ \beta_{\delta}=\mathbb{P}_{\mathcal{H}_{1}}[\delta(\mathbf{N})=\mathcal{H}_{1}]. $$
(24)

In the practical test, parameter σ j is unknown which can be successfully estimated by using MLE (see (21)). Afterwards, we can establish our proposed practical LRT for detecting resampled images. Based on the probability density function (pdf) of hypothesis \(\mathcal {H}_{0}\) and \(\mathcal {H}_{1}\) (20), the Likelihood Ratio (LR) value for the i-th pixel-wise on j-th column vector is given by:

$$ \begin{array}{ll} {\Lambda}_{i,j} (n_{i,j}) =\frac{\Pr\{n_{i,j}\mid n_{i,j}\in \mathcal{H}_{1}\}}{\Pr\{n_{i,j}\mid n_{i,j}\in \mathcal{H}_{0}\}} = \frac{510}{\widehat\sigma_{j}\sqrt{2\pi}}\exp\left( \frac{-n_{i,j}^{2}}{2{\widehat\sigma_{j}^{2}}}\right). \end{array} $$
(25)

Then, it follows from the statistical independence of vectors n i,j that the practical LRT for all the pixel intensities is given by:

$$\begin{array}{@{}rcl@{}} \delta(\mathbf{N})& = & \left\{\begin{array}{ll} \mathcal{H}_{0} \,\text{ if }\, {\Lambda}(\mathbf{N}) = \sum\nolimits_{j=2}^{J-1} \sum\nolimits_{i=2}^{I-1}\log({\Lambda}_{i,j}) \leq \tau_{\alpha}\\ \mathcal{H}_{1} \,\text{ if }\, {\Lambda}(\mathbf{N}) = \sum\nolimits_{j=2}^{J-1} \sum\nolimits_{i=2}^{I-1}\log({\Lambda}_{i,j}) > \tau_{\alpha} \end{array}\right. \end{array} $$
(26)

where the column index j∈{2,...,J−1} in which J denotes the total number of columns, and the decision threshold τ α is the solution of equation \(\mathbb {P}_{0}[{\Lambda }(\mathbf {N})>\tau _{\alpha }]=\alpha \) to guarantee that \(\delta (\mathbf {N})\in \mathcal {K}_{\alpha }\).

5 Experiments

5.1 Performance comparison on single resampled image

To verify the sharpness and effectiveness of our proposed methodology, numerical experiments are illustrated in this section. Prior to our experiments, it is proposed to randomly choose some images from the image database [30], in which all 1338 RAW format images with size 512×384 are converted to TIFF format or JPEG format with different quality factors.

Figure 5 illustrates the results from resampled images by using bilinear interpolation. Compared with the single bright spot from an original image (see Fig. 5c), the up-sampled image with RF equal to 150 %, has a series of bright spots symmetrically surrounding the central point in Fourier domain (see Fig. 5f). It is observed that with increasing the up-sampled factor, the distance between two spots horizontally or vertically is enlarged. This interesting phenomenon is very similar to the pattern of the distance between symmetric peaks in Fourier domain of the 1-D resampled signal. The results of down-sampled images are displayed in Fig. 5l. Compared to the results of up-sampled images, the bright spots surrounding the central point are not very obvious. Let us attribute that characteristic to that down-sampled images lose more information of linear correlation among neighborhood pixels than those of up-sampled images. Thus, it is more difficult to detect a down-sampled image.

Fig. 5
figure 5

Left-most column: comparison between an original image and its corresponding resampled ones (bilinear interpolation); middle column: its corresponding P m a p ’s; right-most column: periodic resampling artifacts in Fourier domain

Full size image

Next, it is proposed to detect a rotated image. Similarly, it can be observed that surrounding bright spots are scattered in Fourier domain (see Fig. 6). Finally, Fig. 7 illustrates the experimental results from another affine transformation. Numerical experiments verify that our proposed algorithm indeed is capable of detecting a resampled image. Besides, we can modify the high-pass filter H for enhancing the difference between the bright spot and its surrounding nuisance noise in order to make peak values more visible.

Fig. 6
figure 6

Left-most column: comparison between an original image and its corresponding resampled one (bilinear interpolation); middle column: its corresponding P m a p ’s; right-most column: periodic resampling artifacts in Fourier domain

Full size image
Fig. 7
figure 7

Left-most column: comparison between an original image and its corresponding resampled one (bicubic interpolation); middle column: its corresponding P m a p ’s; right-most column: periodic resampling artifacts in Fourier domain

Full size image

5.2 Performance comparison on batch of resampled images

To verify our proposed test in a large scale of images, it is proposed to give the experimental performance of our detector for processing a batch of inspected images. We establish an image database of 200 uncompressed gray-level images, which is randomly chosen from Bossbase database [2]. In the following experiments, three practical interpolation methods for image resizing, that are bilinear, bicubic, nearest-neighbor, are used for verifying the effectiveness of our proposed algorithm. It should be noted that our proposed test can still detect other image re-scaling techniques like rotation or shearing. As Fig. 8 illustrates, by using bilinear or nearest-neighbor algorithm for resampling, our detector performs its ability to detect resampled images with small RFs. When nearest-neighbor is applied for interpolating, even resampled images with RF equal to 105 % can be easily detected at the cost of very small FAR. Compared with bilinear or nearest-neighbor, since bicubic partly destroys the correlation of pixels existing in the resampled image, the performance of the detector is degraded. Besides, with increasing RF, the performance of our detector is improved largely. Therefore the larger RF is, the more stronger correlation among pixels exists.

Fig. 8
figure 8

ROC curves comparison by detecting up-sampled images where the legend denotes the resampling factor (RF)

Full size image

Apart from detecting resampled uncompressed images, our proposed detector is capable of detecting resampled compressed images. It is proposed to use dataset one consisting of 500 uncompressed non-resampled images and 500 compressed resampled images. After resampling original uncompressed images by using RF from 75 % to 150 %, all the resampled images are saved as JPEG format with Quality Factor (QF) from 50 to 90 (see Table 1). By increasing QF, JPEG compression nearly does not impact our detection power. Besides, by using different RFs (>100 % or <100 %), our proposed detector performs at the similarly equivalent level. In addition, it is proposed to use dataset two consisting of 500 compressed non-resampled images with QF 75 and 500 double-compressed resampled images. After resampling original compressed images by using RF 105 % and 95 %, all resampled images are saved as JPEG format with QF from 70 to 90 (see Fig. 9). Although double-compression operation to some extent impacts the detection accuracy, the proposed detector performs well by considering its detection rate at the given false alarm rate. Finally, it is proposed to design dataset three consisting of mixed 2000 images which includes 1600 JPEG images with different QFs and 400 uncompressed images. In our experiment, half of this mixed dataset is resampled with RF from 80 % to 120 % while the other half is not resampled. As Fig. 10 illustrates, at the very low FAR, the detection performance of the proposed detector achieves the relevant level. Besides, the resampled images with RF larger than one can be detected more accurately than the images with RF smaller than one. Because resampled images with larger RF has more correlated information existing among pixels than images with smaller RF. Meanwhile, resampling detection in the mixed dataset verifies that our designed test can detect resampled images with mixed compressed and uncompressed format.

Fig. 9
figure 9

ROC curves comparison by detecting resampled double-compressed images in dataset two

Full size image
Fig. 10
figure 10

ROC curves comparison by detecting resampled compressed or uncompressed images in dataset three

Full size image
Table 1 Detection performance comparison at the given FAR α τ =0.05 by detecting resampled once-compressed images in dataset one
Full size table

In practice, the periodic blocking artifact of the JPEG format image possibly collides with the periodic trace of the resampled image. Therefore, when a batch of resampled JPEG format images are inspected, it is very difficult to authenticate whether the periodic pattern is from the blocking artifact or the resampling artifact. Since the artifact is denoted as the only benchmark to detect a batch of images, those artifact-based detectors could not solve this tough problem. In our proposed algorithm, the practical LRT does not use the resampling artifact directly, but instead the ratio of the probability of residual noise, which effectively deals with the problem caused by JPEG compression.

5.3 Performance comparison with state-of-the-art competitors

Some state-of-the-art designed tests based on the bright spots, which are caused by the periodic characteristic of interpolation, have been proposed to detect a batch of resampled images (see [14, 21]). However, in this present paper, it is proposed to design a practical LRT for solving the problem of image resampling detection. To compare our proposed detector with the prior arts, in our following experiments, it is proposed to design three different LRTs based on estimated residual noise n i,j of an image by using the following algorithms: our proposed linear parametric model (15); EM algorithm proposed in [21]; the fixed parameter patternFootnote 4 proposed in [14]. Then, let us compare the detection performance of the designed practical LRT based on three different algorithms. We use all TIFF format images from the image database [30], which are resampled by using bilinear algorithm with 6 different RFs.

Table 2 illustrates the comparison of the detection performance at the give FAR α τ =0.05 or 0.1. With increasing RF, the detection performance of three practical LRTs is improved. Moreover, in the case of using all 6 RFs, the practical LRT designed by the linear model outperforms the other two practical LRTs using prior-art algorithms [14, 21]. Besides, it should be noted that the performance of the practical LRT designed by our proposed linear model achieves 100 % at the given RF equal to 200 %, as well as the practical LRT established by using a fix pattern. We also compare the computation efficiency of the designed practical LRTs based on three algorithms, which is illustrated in Table 3. It can be observed that the LRT designed by the fix pattern performs best due to that the fixed pattern need not estimate the parameters. Since EM algorithm needs iterative computation, the LRT based on it performs less efficiently than two other designed LRTs. Hence, by considering the detection accuracy and computation efficiency, our proposed LRT based on the linear parametric model is the optimal choice.

Table 2 Detection performance (true positive rate) comparison under two hypotheses between \(\mathcal {H}_{0}\) and \(\mathcal {H}_{1}\) at the given FAR α τ ; the proposed test δ (26) designed by using three algorithms; the interpolation algorithm is bilinear
Full size table
Table 3 Comparison of computation time (second) for each inspected image; the interpolation algorithm is bilinear
Full size table

6 Conclusion and discussion

This paper first studies the linear correlation of 1-D resampled signals. By using a linear parametric model, together with the Bayes’ rule, we propose the algorithm of uncovering the resampling artifact. Numerical experiments verify that our proposed algorithm is capable of distinguishing between non-resampled signal and resampled one. Meanwhile, the up-sampled factor of a resampled 1-D signal can be estimated accurately. Moreover, it is proposed to use another linear parameter model and design a test of detecting a resampled image, whose artifact is described as bright spots in Fourier domain. In the case of detecting a batch of images, based on the probability of residual noise extracted from a non-resampled and resampled image respectively, it is proposed to design the practical LRT detector. To compare with LRTs designed by using the prior-art algorithms [14, 21], our proposed practical LRT performs best.

To the best of our knowledge, our designed LRT-based detector is the first test based on hypothesis testing theory for image resampling detection. It can achieve the maximal detection power at the prescribed FAR. Besides, our detector can deal with the problem of authenticating the resampled images from mixed compressed and uncompressed images. In this context, the Uniform distribution describing a non-resampled image is not very accurate. It can be replaced by the more accurate generalized model, which will be studied in the future works.