Keywords

1 Introduction

With the continuous advancement of technology, it is very convenient to get access to various image acquisition devices, and digital images make an important role of lives. However, the emergence and prevalence of a variety of image editing softwares makes it easier than ever before to temper and forge digital images. In recent years, the tampering and forging events have risen a concern about the authenticity and originality of digital images, which is called digital image forensics. As an important branch of digital image forensic, camera source identification focus on the originality of digital images, which can be divided into two branches: (1) model-based, and (2) device-based camera source identification. In this paper, we mainly discuss the latter.

In last decade, several methods were proposed to solve the problem of camera-based source camera identification, such as sensor patter noise [1], sensor dust characteristics [2], and defects in CCDs [3]. The key idea of these methods is to extract unique characteristic features introduced by the hardware of image acquisition devices. Among them, SPN has been widely considered as a reliable fingerprint because of its universality and stability.

Lukas et al. [1] first extracted SPN from digital images using a wavelet-based Wiener de-noising filter. To reduce the false acceptance probability, Goljan et al. [4] proposed the Peak to Correlation Energy ratio (PCE) instead of the normalized correlation to estimate the correlation between the reference and test SPN. Considering computational cost, Hu et al. [5] proposed a new method only taking the large components of SPN into account. Recently, Li et al. [6] proposed an effective feature extractor based on the Principal Component Analysis (PCA) de-noising algorithm [7] to reduce the size of camera fingerprint and significantly improve the performance of several existing methods.

However, the reference SPN extracted through the methods above will be unavoidably affected by scene details, which would significantly influence the accuracy of source camera identification. To solve this problem, Wu et al. [8] introduced an edge adaptive SPN predictor to reduce the influence of images content residual based on content adaptive interpolation, and also proved that the reference SPN with less scene noise could achieve a higher identification accuracy. Recently, Li et al. [9] proposed a framework based on random subspace method (RSM) and majority voting (MV) to suppress the interference of image content.

Different from the methods discussed above, we propose a complexity based sample selection method to obtain more accurate reference SPN. Firstly, we adopted image complexity as the representation of amount of scene details. Then, the sample selection method is used before the extraction of noise residual to select image pathes with less scene detail to generate reference SPN. After the construction of reference SPN, different methods are performed to verify the effectiveness of the proposed method.

The rest of this paper is organized as follows. Section 2 firstly introduce the measurement of image complexity, and then propose the sample selection method based on image complexity. In Sect. 3, experimental results show the effectiveness and applicability of the proposed method, and finally, conclusion is drawn in Sect. 4.

2 Proposed Method

Previous works suggested that SPN should be extracted in several pure images, such as a dark or white one [8]. They insisted that the noise residual extracted from those images with less scene detail is much more pure and reliable. As shown in Fig. 1, the image scene detail obviously differs in different area of an image. In most cases, there are always some regions containing less image content, which means lower image complexity, and the SPN generated from these regions is much more reliable. Inspired by this, a new complexity based sample selection method is proposed in this section, by quantitatively analyzing the image complexity.

Fig. 1.
figure 1

(a) An image taken by Canon_Ixus70. (b) Reference SPN of Canon_Ixus70. (c) The noise residual extracted from (a).

Full size image

2.1 Image Complexity Measurement

There are many kinds of image complexity measurements, in which texture features are widely used and quite effective. Textures are characterized by the relationship of the intensities of neighboring pixels ignoring their color, which means they are a representative pattern of spatial distribution of image intensities. They contain fatal information about the structural arrangement of surfaces and their relationship to the surrounding environment. Since the textural properties of images appear to carry useful information of image content, it is reasonable to consider texture features as the representation of image complexity.

In various methods of texture analysis, the gray level co-occurrence matrix (GLCM) proposed by Haralick and Shanmugam [10] is considered as the most widely used algorithm because of its adaptability and robustness in different scenes. In our paper, the GLCM is adopted to analyze the texture features of the image, considering as the complexity of the image patches.

The co-occurrence matrix is defined over an image according to the distribution of co-occurring values for a given offset. More specifically, GLCM calculates how often a specified pixel with gray-level value occurs either horizontally, vertically, or diagonally to the adjacent pixels. Mathematically, the co-occurrence matrix C, defined over an \(m \times n\) image I and parameterized by an offset \((\varDelta x,\varDelta y)\), is calcultated as:

$$\begin{aligned} {C_{\varDelta x,\varDelta y}}\left( {p,q} \right) = \sum \limits _{i = 1}^m {\sum \limits _{j = 1}^n {\left\{ {\begin{array}{l} {1,~\mathrm{{if }}~I\left( {i,j} \right) = p~and~\mathrm{{ }}I\left( {i + \varDelta x,j + \varDelta y} \right) = q}\\ {0,~\mathrm{{otherwise}}} \end{array}\mathop {}\limits _{} \mathop {}\limits _{} } \right. } } \end{aligned}$$
(1)

where p and q are pixel values of the image, and i, j is the spatial positions in the given image I. What’s more, the offset \((\varDelta x,\varDelta y)\) depends on the direction adopted and the distance at which the matrix is computed. Haralick et al. extracted 14 parameters form GLCM to describe the textural properties of a given image, but some of them show redundancies, and the commonly used parameters are energy, contrast, entropy, homogeneity and correlation coefficients.

As discussed above, the texture features of the whole image can be expressly represented by the combination of those GLCM parameters. However in this paper, our goal is to simply measure the complexity of image patches and find the patch of smallest complexity to extract more pure SPN, rather than precisely describe the sharpness and depth of the texture. So we just select correlation coefficient as the representation of image complexity. The definition of correlation in GLCM is given by:

$$\begin{aligned} cov~\mathrm{{ = }}\sum \limits _{i,j} {\frac{{\left( {i - {\mu _i}} \right) \left( {j - {\mu _j}} \right) G\left( {i,j} \right) }}{{{\sigma _i}{\sigma _j}}}} \end{aligned}$$
(2)

where \(\mu _i, \mu _j\) is the mean of the GLCM elements along the horizontal and vertical direction respectively, and \(\sigma _i,\sigma _j\) is the variance matrix of ith row and jth column.

Correlation coefficient measures the consistency of image textures. Usually, the patch with less scene detail will has a high correlation coefficient. In other words, the higher the correlation coefficient, the lower image complexity. As shown in Fig. 2, it is obvious that the upper left patch of the image, whose correlation coefficient is highest, has less scene detail.

Fig. 2.
figure 2

(a) An image taken by Canon_Ixus70. (b) The image complexity of each patch.

Full size image

2.2 Image Complexity Based Sample Selection

Lukas et al. [1] firstly proposed to extract the reference SPN from digital image by averaging noise residuals. They adopted a wavelet-based Wiener de-noising filter to extract the residual signals from the wavelet high frequency coefficients:

$$\begin{aligned} {r_i} = {I_i} - F\left( {{I_i}} \right) \end{aligned}$$
(3)

where \(I_i\) is the original image and F is the de-noising filter. Then the reference SPN is given by:

$$\begin{aligned} R = \frac{{\sum \limits _{i = 1}^N {{r_i}} }}{N} \end{aligned}$$
(4)

where N is the number of images to generate reference SPN. But the averaging method has an limitation, the image to generate reference SPN are mostly blue sky, which is not realistic in practice.

To address this problem, image patch selection is adopted before the extraction of noise residual. Assuming there are n images denoted by \(I_i~ (i=1,2,3,...,n)\) taken by the same camera. Firstly, every image is segmented into several non-overlapping blocks with the size of \(256 \times 256\) pixels. Then, the complexity coefficient cov is calculated using Eq. 2 of each image block to form a complexity matrix, as shown in Fig. 2. For all images in the experiment database, complexity coefficients of the same location are then added together to form a new complexity matrix N. For the purpose of using the region with less scene detail to generate the reference SPN, the block with maximum value in N is selected to extract noise residual for construction of reliable reference SPN using method in [1]. The block diagram of complexity-based SPN generation is shown in Fig. 3.

Fig. 3.
figure 3

The block diagram of complexity based sample selection.

Full size image

3 Experimens and Results

3.1 Experimenal Setting

In order to evaluate the performance of the proposed method, we carry out our experiments on the Dresden mage Database [11]. A total of 1200 images from 10 cameras are considered in the experiments, which are all taken indoors and outdoors with variety of natural scene sight. The 10 cameras devices belong to four camera models, and each model has 2–3 different devices. Table 1 lists the details of the 10 cameras used in the experiments.

Table 1. Camera used in the experiments.
Full size table

For each camera, 120 images are divided into two subsets for training and testing, of which 50 images are used to construct the reference SPN, and the test set is consisted of the remaining 70 images. For a fair comparison, the basic SPN [1], PCA [6]and RSM [9] method mentioned above are respectively performed with and without the proposed sample selection method. Considering the computational cost, the experiments without sample selection are performed on the central block with the size of \(256 \times 256\) cropped from the original image.

3.2 Performance Evaluation

As described in Sect. 2, the image complexity based sample selection method is performed before the extraction of SPN. We first cut the images into the size of \(2048 \times 2048\) from the upper left corner of the original images. Then, the cropped images are segmented into 64 patches with size of \(256 \times 256\). For all images in the experiment database, correlation coefficients of the same locations are then added together to form the complexity matrix of whole database, which is shown in Table 2.

Table 2. Correlation coefficients of image database.
Full size table

As we can see from Table 2, the patch of 1st row and 2nd column has the largest correlation coefficient, which also means minimum image complexity and less scene detail. So, we select this patch as the representation of images to extract SPN. In order to verify the effective of the proposed method, we carried out several comparative experiments with and without sample selection, and the experimental results are shown in tables below.

The basic SPN method of image source identification is to directly extract noise residual from several images to form reference SPN by averaging. Then, the correlation coefficient between the reference and test noise residual is calculated. The test image is belong to the origin whose correlation coefficient is highest. The experimental results are shown in Tables 3 and 4. When adopted the proposed sample selection method, the average identification accuracy of basic SPN method has an improvement of 0.42% from 87.29% to 87.71%. The average identification accuracy of three other methods are also listed in Table 3.

Table 3. Average identification accuracy with and without sample selection.
Full size table
Table 4. Identification accuracy of basic SPN with and without sample selection (%).
Full size table

PCA is a widely used method to reduce the dimension of camera fingerprint while represent the original data as well as possible. When selecting an appropriate proportion of principal components, there should be a certain extent improvement of identification accuracy. The results of the experiment are shown in Table 5. It is easy to find that the identification accuracy of SS PCA has an obvious improvement compared with the PCA without sample selection, especially when the proportion of principal components G(T) is smaller. When G(T) = 0.99, the SS PCA shows its highest accuracy of 88.14%, and the average improvement of identification accuracy is 0.78%. Table 5 also lists the identification accuracy of each device respectively when G(T) = 0.99.

Table 5. Identification accuracy of PCA SPN with and without sample selection (%).
Full size table

In work [9], random subspace method and majority voting is used to improve the identification accuracy. There are two parameters which would affect the identification performance: the dimension of each random subspaces M and the number of random subspaces L. According to [9], we empirically set \(L=600\) while M/d various from 0.1 to 1, where d is the size of entire feature space. It is worth mentioning that the performance of RSM method is the same as that of basic SPN method when \(M/d = 1\), which has been proved by the results shown in Table 6. The highest identification accuracy of RSM method and SS RSM method is 87.29% and 88.14% respectively, and the average improvement is 0.84%. Similar with Table 5, the improvement is more obvious when M/d is smaller. Also, Table 6 shows the identification accuracy of each device with highest average identification accuracy when \(M/d=0.6\) (for RSM) and 0.3 (for SS RSM).

Table 6. Identification accuracy of RSM SPN with and without sample selection (%).
Full size table
Table 7. Identification accuracy of PCA RSM with and without sample selection (%).
Full size table

Then another experiment combining sample selection, PCA and RSM with \(L=600\) and G(T) = 0.99 is performed to see whether the identification accuracy would be further improved. Table 7 shows the experimental results. The highest accuracy of SS PCA RSM method is 88.14%, whose average accuracy improvement is 0.57%. Table 7 also shows the identification accuracy of each device with highest average identification accuracy when \(M/d=0.6\) (for PCA RSM) and 0.8 (for SS PCA RSM).

4 Conclusion

An effective complexity based sample selection method for camera source identification is proposed in this paper. Considering the noise residual extracted from natural images may be seriously affected by scene details, we adopt a sample selection method based on image complexity to select image patches with less image content to generate reference SPN, which could significantly enhance the credibility of reference SPN. An extensive comparative experiments are carried out to prove the effectiveness of the proposed method. And the experimental results show that the proposed method can eliminate the influence of image content and improve the identification accuracy of the existing methods.