Abstract
Salient Object Detection (SOD) in natural images is an active research area with burgeoning applications across diverse disciplines such as object recognition, image compression, video summarization, object discovery, image retargetting etc. Most salient object detection methods model this problem as a binary segmentation problem where firstly a saliency map is found which highlights the salient pixels and suppresses the background pixels in an image. Secondly, some threshold is applied to obtain the binary segmentation from the saliency map. Thus, thresholding is an important ingredient of salient object detection methods and affects the SOD performance. In this paper, we provide a comprehensive review of various thresholding methods in literature employed for SOD. We have developed a taxonomy of thresholding methods which shall be useful to the researchers and practitioners working in this fascinating research field. Further, we also discuss unexplored thresholding approaches which can be employed in SOD. Various existing and proposed performance measures to analyze SOD methods that depend on thresholding are also presented. Experiments on popular thresholding methods have also been carried out to show the dependence of qualitative and quantitative performance on thresholding.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Humans can understand complex natural scenes without much effort. This capability of human beings is computationally implemented with the help of Salient Object Detection (SOD). The term ’Salient’ refers to those areas in an image or video which are distinctive when compared to its surrounding. SOD in natural images has become an active research area in the past two decades due to its enormous applications in areas such as object recognition [145], image resizing [6], image retargetting [40], image and video compression [42, 50], image thumbnailing [117], video summarization [53, 112], photo collages [150], image quality assessment [98], small device displays [20], image segmentation [135], image editing and manipulating [41, 118], image retrieval [159], object discovery [33, 66], human robot interaction [167] etc.
The origins of salient object detection lies in feature integration theory (FIT) proposed by Treisman and Gelade [173]. In this theory, it is pointed out that visual features derive human attention and help the task of searching. This concept was furthered by Koch and Ullman [75] who developed a feed forward neural network model and introduced the concept of saliency map. This map represents the attended locations in an image. The pioneering work on salient object detection was done by Itti et al. [51] who implemented the first salient object detection framework on natural as well as synthetic images. They also suggested that saliency can be computed in two ways i.e. (i) Bottom-up and (ii) Top-down. Bottom-up methods employs low level image features such as contrast, color, edges, orientation etc. for computing the saliency map of an image. Bottom-up methods are stimulus driven, fast and are independent of the task at hand. On the other hand, Top-down methods are based on high level cognitive characteristics such as knowledge, expectation etc. Top-down methods are slow, volition-controlled and task dependent. There are some methods [125, 132] in literature which employ both bottom-up and top-down methods for salient object detection and can be put in the category of mixed methods. Studies in human visual system [162] have also shown that both bottom-up as well as top-down mechanism are necessary for proper functioning of human visual attention system. A comprehensive survey of salient object detection methods has been done by Borji and Itti [12]. They have broadly divided salient object detection models into eight categories [12] viz. (i) Cognitive Models [51, 125, 177] (ii) Bayesian Models [96, 200] (iii) Decision Theoretic Models [39] (iv) Information Theoretic Models [17, 48] (v) Graphical Models [7, 45, 100, 101] (vi) Spectral Analysis Models [3, 43, 47] (vii) Pattern Classification Models [24, 62, 68, 132] (viii) Other Models [142, 143]. There are some methods which lie in more than one category. However, most of the salient object detection methods are derived from cognitive models directly or indirectly.
Once saliency map has been obtained using some salient object detection method, the next step is to get the binary attention mask corresponding to the saliency map by employing some threshold. Thresholding is an important image processing operation having several applications [157] such as document image analysis [1, 63], map processing [174], target detection [8], defect detection in materials [158] etc. In thresholding, it is presumed that there is significant difference between the gray levels of some object in the image and the background. The image containing some object can be converted to a binary image by employing some threshold. In image processing context, a threshold is a gray level which can partition the pixels in an image into two classes i.e. (i) pixels whose values are greater than threshold and (ii) pixels whose values are less than threshold. The pixels whose value is equal to threshold can be put in any one of the two categories. Thus, thresholding converts a gray level image into a binary image. The pixels greater than threshold can be given label 1 and pixels less than threshold can be given label 0 or vice-versa depending upon the application. A comprehensive survey of more than 40 image thresholding techniques is done by Sezgin and Sankur [157]. They have categorized various thresholding methods into six categories depending upon the information used for computing the threshold. Each of these six categories is discussed briefly below:
-
Histogram Shape based: Thresholding methods under this category are based on the histogram shape properties. In this category, methods usually search peaks and valleys for determining the threshold. Rosenfield and Torre [148] analyzed the histogram concavities and the convex hull and suggested that the concavities with minimum values can be used as threshold. Sezan [156] convolved the histogram of an image with smoothing and differencing kernel and analyzed the peaks in the histogram. the threshold is determined to be between the first and second zero crossings.
-
Clustering based: In these thresholding methods, the gray level image data are always clustered into two. The threshold is found to be mid point of the two peaks of the histogram in research works [83]. Some researchers have fitted Gaussian Mixture Models (GMMs) to the histogram for determining the threshold [72]. A modified form of clustering called mean-square clustering is suggested by Otsu [127] and fuzzy clustering is suggested by Jawahar et al. [52] for determining the threshold.
-
Entropy based: These thresholding methods employ entropy measure of an image for determining the threshold. Johannsen and Bille [60] and Pal et al. [128] proposed the seminal work in this category. Threshold is determined based on the idea that maximum information transfer is indicated by maximization of entropy measure in the thresholded image [65, 133, 134, 152]. In the research work [87], threshold is determined by minimizing the cross entropy between the original gray level image and the thresholded image. In recent years, entropy based thresholding has received attention from the researchers in various fields. Mahmoudi and Zaart [114] have carried out a recent survey of entropy based thresholding techniques.
-
Attribute Similarity based: These methods exploit the similarity of some measure between the original image and the binarized image. The attributes can be gray-level moments edge matching, stability of segmented objects, shape compactness, texture etc. In some research works, the similarity of features between original and binary images are measured using fuzzy measure [122, 141] or cumulative probability distributions [32] or the amount of information gained after segmentation [84]. A popular method proposed by Tsai [175] in this category hypothesize that gray image is a blurred binary image. The thresholding is computed in such a way that first three gray level moments of original and the binary image match.
-
Spatial Thresholding Method: In this category of thresholding methods, not only the gray level information is exploited but also the relationship of a pixel among its neighboring pixels is also considered. One of the earliest methods under this category is suggested by Kirby and Rosenfeld [71] in which local gray levels are used for computing the threshold. This was followed by other researchers in other improvements such as using relaxation for improving binary maps [31], enhancement of histogram using Laplacian of image [183], quadtree thresholding [185], etc.
-
Locally Adaptive thresholding: In these thresholding methods, threshold is computed at each pixel depending upon local pixel characteristics. Nakagawa and Rosenfelt [123], Deravi and Pal [25] were among the early researchers in this category. In research works [126, 153] local variance is employed while in [184, 194] local contrast is employed. Palumbo et al. [129] suggested a centre-surround scheme for computing the threshold.
From the above discussion, it is apparent that thresholding is an important image processing operation. It is also extensively used in salient object detection to obtain binary attention mask from the saliency map. However, there is no research work in literature which can give insights to various thresholding methods used in salient object detection. In this paper, we have developed a novel taxonomy of various thresholding methods used in literature for salient object detection. We have considered various factors while forming this taxonomy as discussed in Section 2.
The rest of the paper is organized as follows: Section 2 introduces the general concept of thresholding in SOD context and a novel taxonomy is presented. Section 3 to 6 describe in detail the methods under proposed taxonomy. In Section 7, we present existing and proposed performance measures for SOD methods which depend on thresholding. In Section 8, we have also presented a comparative study of popular thresholding methods in salient object detection. Discussion and unexplored thresholding methods are presented in Section 9. Concluding remarks and future directions are given in Section 10 at the end.
2 Thresholding in salient object detection
As we know, the major objective in salient object detection is to determine a binary attention mask which can be employed for extracting the object of interest from a digital image. Most SOD methods generate a saliency map which is required to be converted into a binary map. In this binary image, foreground is represented as label 1 and background is represented as label 0 or vice versa depending upon the application.
Now suppose we are given a digital image I(x, y) of size W × H where x ?{1, 2,..., W} and y ?{1, 2,..., H} and S(x, y) is the corresponding saliency map. Some thresholding is applied on the saliency map S to convert it into a binary attention mask A(x, y). Figure 1 shows a sample image with its corresponding saliency map, binary attention mask corresponding to the saliency map and the ground truth of the image. Ideally, the thresholded version should be as close to the ground truth as possible.
From the above discussion, it is apparent that thresholding is an indispensable step in most salient object detection methods. Here, we present a comprehensive survey of thresholding methods employed in salient object detection keeping in view the following factors:
-
1.
Whether threshold is applied by SOD method or not?
-
2.
If a threshold is applied then whether a single, multiple or a combination of thresholds is applied on the image?
-
3.
Whether thresholding is image dependent or not?
-
4.
Whether thresholding requires human intervention or not?
Few SOD methods produce only the saliency maps and not the binary attention masks. On the other hand, most salient object detection methods produce binary attention mask by employing some threshold on saliency map. This threshold can be found in several ways. To understand various thresholding methods employed in SOD in a better manner and considering the above mentioned factors, in this paper, we have developed a simple taxonomy of various thresholding methods in salient object detection as shown in Fig. 2. We broadly divide the thresholding methods employed in salient object detection in five categories viz. (i) No Threshold (ii) Global Thresholding (iii) Local Thresholding (iv) Hybrid Thresholding and (v) Other Thresholding Methods. Methods falling under each of these categories are discussed in details in the following sections.
There are few SOD methods whose objective is to find salient regions in an image and output only the saliency map. Thus, there is no requirement of thresholding in such cases. The performance of these methods is usually analyzed from the quality of saliency map generated by the methods. This saliency map is simply a grey level digital image which highlights the object of interest i.e. salient object and suppresses the background. Furthermore, the saliency map generated using some method is compared to other saliency map subjectively. SOD methods falling under this category together with features employed by each method are listed in Table 1.
As no thresholding is applied on saliency map to obtain the attention mask, these methods possess least time complexity among all other thresholding approaches. In addition, the saliency map obtained can be used for applications including edge or contour detection. The disadvantages of these thresholding methods include (i) lack of objective performance measures and (ii) difficulty in choosing better method among close competitors.
3 Global thresholding
Salient object detection methods under this category employ Global or single threshold (T) for converting the saliency map into binary attention mask using the following equation:
The pixels in the saliency map having values greater than the threshold are marked as foreground while the pixels having value less than the threshold are marked as background. It is inevitable to consider how this single threshold is determined. One of the simplest way to determine the threshold is to try all possible values of threshold for all the images at hand and choose the one which gives best performance. However, it leads to higher computational requirements. Thus, we need to look for some other computationally efficient way of finding this threshold which can give better performance and is computationally efficient. Various researchers have proposed several global thresholding methods from time to time.
In global thresholding methods, an important factor to consider is whether threshold depends on the image at hand or not. The image dependent threshold varies with respect to each image while image independent threshold remains invariant to all the images considered at some instance of time. Another factor to consider is whether human intervention is required or not. Based on these factors, global thresholding is further divided into three subcategories : (i) Fixed thresholding (ii) Semi-Adaptive thresholding and (iii) Adaptive thresholding.
3.1 Fixed thresholding
SOD methods falling under this category employ image independent thresholding and do not use any image characteristics to determine the threshold. In these methods, threshold is fixed to some value between [0, 255] or [0, 1] depending upon the range of saliency map normalization. Thresholding methods under this category are further divided into two classes: (a) Single Fixed Thresholding (b) Range of Fixed thresholding.
In Single Fixed thresholding, only one threshold irrespective of the image is employed. The value of the threshold lies between T ? [0, 255] o r [0, 1]. The methods which employ single fixed threshold are given in Table 2.
In Range of Fixed Thresholding methods, whole range of possible thresholds with suitable intervals is employed in the range [0,255] or [0,1]. In the range [0,255], the threshold is usually varied from 0 to 255 in steps of size 1 thereby producing 256 attention masks. In the range [0,1], the threshold is varied from 0 to 1 usually in steps of 0.01 resulting in 101 attention masks. This type of thresholding is widely used in literature due to its application in drawing Receiver Operating Characteristics (ROC) curve, Precision-Recall (PR) curve and computing the Area Under the ROC Curve (AUC) performance measures. ROC curve is drawn between true positive rate (TPR) and false positive rate (FPR) at each threshold from 0 to 255. Methods employing fixed range thresholding are listed in Table 3 while in few Fixed Range thresholding methods [59, 64, 92, 178, 182] the threshold is varied between 0 and 1.
3.2 Semi-adaptive thresholding
In this category, the threshold consists of two components: (i) one component set by the user and (ii) other component computed from the image. One of the methods in this category is proposed by Jin et al. [59]. In their method, the threshold T is computed using the following equation:
where a =?0.70 and s is the variance of the image.
In these thresholding methods, manual intervention is required to set the value of parameter a. This value can be varied to tune the performance of salient object detection for the application at hand. However, these methods have some drawbacks such as (i) not fully automatic (ii) poor generalization. Therefore, this type of thresholding is not very popular in literature.
3.3 Adaptive thresholding
This is the most widely employed thresholding method in salient object detection. In this kind of thresholding, a single global threshold (T) is determined using image characteristics. Due to this dependence, adaptive thresholding is image dependent i.e. it varies with each individual image. To determine the adaptive threshold, various thresholding functions or algorithms are employed by SOD methods as given in Table 4.
Hou and Zhang [47] set the threshold to be thrice the average saliency value. The threshold T is computed with the help of following (3):
Another most popular threshold is suggested by Achanta et al. [3] in literature. The proposed threshold is twice the average saliency value as given in (4). This threshold is used by several researchers subsequently as shown in Table 4 .
Judd et al. [62] suggested to use such threshold which can highlight some percentage of image pixels (e.g. 1,3,5,10,15,20,25,30) to be salient. Rosin et al. [149] employed Tsai’s moment preserving algorithm [175] for determining the global threshold. In Tsai’s algorithm, a grey level image is assumed as a blurred version of an ideal binary image. In order to find the ideal unblurred version of the image i.e. binary image, first three moments are preserved. Thus, these three moments are same in grey level image and the binarized version of the image. The i - t h moment (m i ) of an image I is defined as below:
Alexe et al. [4] first resized the saliency map to 64 × 64 resolution and then found the threshold to be the average saliency value at this re-scaled saliency map. Otsu algorithm [127] has also been employed by various researchers such as Khuwuthyakorn et al. [67], Luo et al. [109], Liang et al. [93], Hu et al. [49] etc. Otsu algorithm is a clustering based algorithm to determine the global threshold. In this method, a global threshold is determined such that the spread between the foreground pixels and the background pixels is maximum while the combined intra-class spread is minimum.
Yu et al. [196] proposed using a threshold which is a t times the average saliency value. The value of a t is set to a low value in order to get high recall rate. Jia et al. [54] and Scharfenberger et al. [154] suggested using a threshold which is sum of two quantities i.e. mean (m) and standard deviation (s) of the saliency map.
Jiang et al. [57] and Zhao et al. [210] employed a threshold which is obtained by multiplying the average saliency value with a factor of 1.5. This multiplication factor was varied between [0.1,6] by Wang et al. [181] and it was set to 1.7 by Xu et al. [188]. Similarly, Xu et al. [189] introduced another factor ? to be multiplied with the average saliency value for determining the threshold. A pixel in the saliency map (SMAP) is termed as a salient if its value is more than ? ×mean(SMAP), otherwise it is marked as background. Singh et al. [165] and Arya et al. [5] have suggested a two step thresholding. In the first step, object’s silhouette’s are determined using Canny edge operator [18]. Subsequently, mean value of the saliency map corresponding to object’s silhouette is used as threshold. Guo et al. [44] have suggested using median value of the saliency map as the threshold.
Gao et al. [37], Zhang et al. [198], Lu et al. [108], Liu et al. [99], Zhou et al. [215] used the average saliency value as the threshold while Wang et al. [179] set the threshold to be half the maximum saliency value. Ren et al. [145] have suggested saliency cut algorithm for converting the saliency map into binary attention mask. In saliency cut algorithm, an attention window AW of size 10 × 10 is first initialized such that the center of the window is the position where mean saliency value in the window in maximum. This window is then extended in both x-direction and y-direction until the pixels on the boundary are smaller than a fixed threshold. Zhou et al. [212] set the threshold to be equal to maximum of the twice of the average saliency value and the maximum grey level value. Kumar et al. [78] have employed negative transform of the mean saliency value as threshold. In their research work, the saliency map was normalized between 0 and 255.
The Global thresholding methods have the major advantage of being simple to implement as only a single threshold is employed throughout the saliency map. The time complexity of these methods is more than No Threshold methods but less than local thresholding methods. The performance measures such as ROC curve and PR curve can only be drawn with the help of range of fixed thresholding which is a Global thresholding method. The disadvantage of global thresholding is that no neighbourhood information i.e. local structure of the image is used in determining the threshold. Single fixed thresholding also has poor generalization across different images as can be observed from Fig. 3. Fixed threshold may be good for some images and bad for other images. Moreover, single global threshold may not be suitable for all the regions in an image. Semi-Adaptive thresholding requires human intervention which is unwanted for fully automatic salient object detection.
4 Local thresholding
In local thresholding methods, multiple thresholds are computed and applied in a local manner. These local thresholds may be based on a superpixel or block. In this paper, we assume that superpixels are group of pixels that can have arbitrary shapes while blocks are rectangular patches consisting of pixels. Local methods are computationally expensive than global thresholding methods. We categorize Local thresholding methods into two claases: (i) Superpixel based and (ii) Block based.
4.1 Superpixel based
Chang et al. [19] suggested using the global adpative thresholding same as Achanta et al. [3] but applied it in a local manner on the superpixels. Given m superpixels of size a m , the threshold is computed using the following equation:
where d m is the number of eye fixations in the super pixel.
4.2 Block based
Li et al. [90] have suggested another local patch based thresholding method in which the saliency value of a patch is multiplied by some constant depending upon the variance of the patch. In this method, a patch p k is determined to non-salient or low-salient with the help of following equation:
Here s 1 and s 2 are the thresholds which determine whether a patch is non-salient or low-salient. Depending upon these thresholds, patch p k is multiplied with some constants to modify the saliency value of all the pixels in that patch. These constants viz. µ 1 and µ 2 are chosen such that 0 < µ 2 < µ 1 <?1 (Table 5).
The advantages of Local thresholding methods is that local neighbourhood structure is used in determining multiple thresholds in an image. Furthermore, local thresholding methods are usually insensitive to global changes such as illumination. Local thresholding also suffers from some drawbacks such as the time complexity of these methods is more than global thresholding methods. Moreover, determining optimal patch size for threshold determination is another problem in these methods. These methods do not exploit global image characteristics. These methods may also suffer from blocking phenomenon. Local thresholding methods are scarcely used in literature due to several disadvantages.
5 Hybrid thresholding
Hybrid thresholding employ both local as well as global thresholding methods for converting the saliency map into attention mask. In this category, either a combination of local and global methods or region growing is employed for determining the final threshold(s) or for obtaining the binary attention mask from the saliency map. Based on these criteria, hybrid thresholding methods are divided into two categories: (i) Local and Global thresholding (ii) Region growing/Optimization thresholding (Table 6).
5.1 Local and global
In this thresholding, both local and global thresholds are computed using the saliency map. The local threshold is computed from local patches while global threshold is computed from the whole saliency map. These patches are then marked as foreground or background using some mechanism. In research works [76, 88, 160, 172], first the image I is over-segmented using mean-shift algorithm followed by computing the average saliency value p i for each segment (say s i ) and m is the overall mean saliency value of I. Then s i is marked as foreground if p i >?2 × m.
5.2 Region growing
Ma and Zhang [111] suggested using fuzzy growing for segmenting the saliency map into binary attention mask. In their research work, saliency map is modeled as a probability space with regard to fuzzy events corresponding to attended U f and unattended U b regions, the fuzzy membership function of these events is defined as below:
where x denotes the gray level of the saliency map. Here, a and u are the parameters which are determined using a modified minimal difference of entropy metric.
Mehrani et al. [120] have suggested using binary graph-cut optimization for converting the saliency map into binary attention mask. In their research work, initial segmentation is refined using a binary graph-cut optimization based trained classifier. This initial segmentation thus obtained is supplied to Iterative graph cut algorithm [15] which renders the final attention mask.
Marchesotti et al. [117], Luo et al. [109], Murato et al. [121] and Xie et al. [187] have employed Iterative graph cut algorithm for finding the binary attention mask from saliency map. Marchesotti et al. [117] suggested obtaining the initial binary map for iterative graph cut algorithm in one of the two ways:
-
(i)
binary map S b using a hard threshold t h b i n which is set to 0.6 in their research work.
-
(ii)
binary map S g using two thresholds i.e th+ and th-. The pixels above th+ are labeled as foreground pixels and pixels below th- are marked as background pixels. The pixels lying between th + and th- are assigned to foreground or background based on an energy minimization function.
The initialization of the iterative graph cut algorithm is done with binary map S * as follows:
where t h d is set to 0.1.
Luo et al. [110] suggested global salient information maximization (GSIM) for saliency computation and employed the output of GSIM for initializing the iterative graph cut algorithm. Muratov et al. [121] and Xie et al. [187] employ generic iterative graph cut algorithm without any initialization.
Cheng et al. [22], Jian et al. [56] and Ju et al. [61] have employed fixed threshold value of 70 to find an initial binary attention mask. This attention mask is then supplied to Grab-cut algorithm for segmentation. Zhang et al. [199] and Zhao et al. [211] have employed Grab - Cut algorithm for segmentation of the saliency map. The main drawback of Grab-cut algorithm is that it is interactive and user is required to draw a rectangle for initializing the algorithm.
Hybrid thresholding possesses the advantage of exploiting local as well as global information from the saliency map for determining the threshold. However, the complexity of these methods is more than local and global thresholding methods. In some hybrid thresholding methods, manual intervention is required which is undesirable. Moreover, region growing methods may suffer from local minima.
6 Other thresholding methods
Some thresholding method which do not fall under any of the above categories are classified under this category. Two of the popular thresholding methods used by Itti et al. [51] were Winner take all (WTA) and Inhibition of Return (IOR) [74]. Salient locations are detected in an image based on the decreasing saliency values. Conditional Random Field (CRF) [80] is another way to determine the thresholding in salient object detection. In a CRF model, linear weights for combining multiple features maps is learnt through a training set. Simultaneously, CRF also partitions the image pixels into foreground and background. CRF is based on conditional probability and minimizing a energy function. This energy function is defined in different ways by various researchers. These thresholding methods are not popular in literature.
7 Performance measures
Once a thresholding method has been chosen for an algorithm, the next critical step in salient object detection is measuring the performance of SOD method(s). A method can be shown to outperform other methods if its performance is better in comparison to other methods. In this section, we present a brief review of the popular performance measurement criteria. Performance of the SOD methods can be measured in qualitative and quantitative terms.
7.1 Qualitative measures
The performance of SOD methods is measured in terms of qualitative measure by comparing the saliency maps produced by SOD methods. This type of performance measure is employed to visually show the quality of produced saliency map. A drawback of qualitative measure is that it is subjective and varies across different people.
7.2 Quantitative measures
In contrast to qualitative measures, quantitative measures assign numerical values to performance. Quantitative measures are also objective performance measures. For a given image I, suppose S is the corresponding saliency map produced by some method. If G denotes the ground truth corresponding to I, then a confusion matrix corresponding to pixel wise saliency map is computed as given in Table 7.
Based on this confusion matrix, various performance measures are computed such as Precision, Recall and F-Measure etc. Precision is defined as the ratio of actual foreground pixels (TP) out of the foreground pixels found by the method. Recall is defined as the ratio of actual foreground pixels found (TP) out of the foreground pixels (TP + FN) in the ground truth.
There is always a trade off between P r e c i s i o n and R e c a l l. e.g. One can achieve high P r e c i s i o n or high R e c a l l rate simply by setting appropriate threshold. These two performance measures are therefor combined into another performance measure called F - M e a s u r e in literature [146]. The traditional F - M e a s u r e called F 1 - M e a s u r e is defined as follows:
F 1 - M e a s u r e gives equal importance to P r e c i s i o n and R e c a l l. A more general F - M e a s u r e called F ß - M e a s u r e is define in literature as follows:
Here, the parameter ß controls the relative importance of P r e c i s i o n versus R e c a l l. The value of ß ? [0,1) gives more importance to P r e c i s i o n than R e c a l l while the value of ß =?1 gives equal importance to both P r e c i s i o n and R e c a l l (aka F 1 - M e a s u r e) while the value of ß >?1 gives more importance to R e c a l l than P r e c i s i o n.
All the above performance measures require that a particular threshold is fixed based on which these can be computed. This problem can be overcome if we set every possible threshold (e.g. 0 to 255 if saliency map is normalized between [0,255]). False Positive Rate and True Positive Rate are then computed for each value of threshold and plotted in a 2-D graph. The methods whose performance is to be compared are plotted in a single graph called Receiver Operating Characteristic (ROC) curve. The method whose curve is on the top left corner of the ROC curve is declared best. It is possible that two or more ROC curves cross other ROC curves thereby rendering it difficult to say which method performs best. To overcome this problem, a new performance measure called Area Under the ROC curve (AUC) is defined whose value renders a numerical comparison between methods. Similar to ROC curve, there is another curve called Precision-Recall (PR) Curve in which for every possible Recall, the value of Precision is computed and then plotted in a single graph.
Besides the above popular thresholding performance measures, there are other performance measures which are used in image processing literature [157] but have not been used in salient object detection. Some thresholding measures which can prove useful in measuring performance of SOD models are (i) Misclassification Error and (ii) Relative foreground area error. We have redefined these measures in terms of data available in confusion matrix.
- (i) Misclassification Error (ME)::
-
This performance measures the fraction of false positive (FP) and false negative (FN) relative to the actual pixel class. ME can be defined mathematically as given in (15):
$$ ME = \frac{FP+FN}{TP+FN+FP+TN} $$(15) - (ii) Relative Foreground area error::
-
This performance measure was originally defined by Zhang et al. [197] and called relative ultimate measurement accuracy (RUMA). This performance measure was modified by Sezgin et al. [157] for foreground area of an image. This performance measure is define as follows:
$$ RAE = \left\{\begin{array}{lllllllll} \frac{FN}{TP + FN} & \text{if} \; TP < P \\ \frac{FP}{TP + FP} & \text{otherwise} \end{array}\right. $$(16)where P is the foreground area in the ground truth of image I and TP is the area of foreground image in the thresholded image I '.
8 Performance evaluation
To choose a thresholding method, one has to consider various aspects before choosing a thresholding method. Here, we present a comparison of various thresholding methods in terms of time complexity, quantitative and qualitative performance.
8.1 Time and space complexity
The amount of time required by a thresholding method is proportional to the number of basic operations performed for computing the threshold and then applying this threshold on the saliency map. The amount of space required is directly proportional to the number of thresholds to be applied on the image. We give a general comparison of time and space complexities among different thresholding categories. In No Thresholding, SOD methods output only the saliency maps and hence no operation is involved for obtaining the attention mask. In global thresholding methods, only a single threshold is applied throughout the saliency map for obtaining the attention mask. In Local thresholding methods, multiple thresholds are computed at super-pixel or block level to convert the saliency map into attention mask. Hybrid thresholding requires both local and global thresholds or region growing for finding the attention mask. As region growing is an optimization problem, which requires more computation. Hence, the thresholding categories according to time and space complexities can be put in the following order:
In Fixed thresholding methods, there is no need for computing the threshold explicitly. Hence, these methods have O(1) complexity. Adaptive thresholding methods exploit the saliency map for computing the global threshold. If n (= W * H for image of size W × H) is the total number of pixels in the saliency map of an image, then the time complexity of popular global adaptive thresholding methods is given in Table 8 below:
The Semi-adaptive thresholding method proposed by Jin et al. [59] involes computation of standard deviation of setting of a constant. Thus, time complexity of Jin et al. [59] method is O(n). Local thresholding methods involve computation of multiple thresholds on patches of the saliency map. Let p is the number of local patches of average size n p . Thus, Chang et al. [19] have time complexity of O(p n p ). Similarly, Li et al. [90] applies two fixed thresholds s 1 and s 2 to define low saliency regions and non-salient regions respectively. Similar to Chang et al. [19] method, the time complexity of Li et al. [90] method is O(p n p ). Hybrid thresholding methods segment the saliency map into attention mask based on global and local thresholding or optimizing some objective function. We have already discussed the time complexity of global and local methods. The hybrid methods which optimize some thresholding function requires a number of iterations (T) for convergence. The time complexity of mean shift algorithm is O(Tn 2) while for other threshoilding methods such as Grab-Cut or Iterative graph cut algorithm, time complexity is O(Tn 3). The thresholding methods under Other Thresholding require some training images to learn some parameters and then produce the attention mask. It is difficult to directly compare these methods with thresholding methods in other categories.
8.2 Quantitative evaluation
In this paper, we have presented more than 30 thresholding methods. Comparing all of them here is out of scope of this paper. However, more than half of thresholding methods in salient object detection lie in global adaptive thresholding. This is due to the fact that global thresholding is simple to implement and these methods compute the threshold value from the saliency map of an image. Here, we present quantitative comparison of popular thresholding methods. The experiments are performed on ASD [3] and ECSSD [161] datasets. Each of these datasets have 1000 natural images and the corresponding ground truth images. Some example images from both these datasets are shown in Fig. 4. All the images in the datasets are of size 300 × 400 or 400 × 300.
The saliency map for all the images was obtained using Liu et al. [101] method and the performance was compared in terms of Precision, Recall and F-Measure. The performance comparison of popular thresholding methods is shown in Figs. 5 and 6 for ASD and ECSSD datasets respectively.
It can be readily observed from Figs. 5 and 6 that the thresholding using Yu et al. [196] gives maximum recall among other methods. This is due to the fact that in the thresholding function employed in Yu et al. method, a t is set to a small value in order to have better recall. Further, threshold functions used by Achanta et al. [3] and Kumar et al. [78] give best precision and F-Measure among all the compared methods on both the datasets. It is also worth noting that thresholding certainly affects the performance of salient object detection methods and hence supports the belief that thresholding is an important step in any SOD methods.
8.3 Qualitative comparison
As apparent from previous discussion that thresholding affects the performance of SOD methods. Here, we present qualitative (or visual) comparison of various popular thresholding functions given in Figs. 5 and 6.
Figure 7 shows two sample images from ASD [3] dataset and their corresponding saliency maps obtained using Liu et al. [101] method. These saliency maps are then converted into binary attention by employing thresholding methods as given in previous subsection. From Fig. 7, it can be seen that the binary attention masks for both the images in (c), (h) and (m) are of poor quality. The binary attention masks produced by (e), (f), (g), (j) and (l) are better than other attention masks. However, among these better attention masks, it is difficult to say which one is better. The visual comparison also confirms the dependence of performance of SOD methods on thresholding.
Figure 8 shows two sample images from ECSSD [161] dataset and their corresponding saliency maps obtained using Liu et al. [101] method. From Fig. 8, it can be readily observed for both images, the binary attention mask in (h), (m) and (n) are not of good quality. The binary attention masks produced by (c), (i), (o) and (p) are better than other attention masks. Th eattention masks in (k) and (l) are better for top image and poor for bottom image. However, among the better attention masks, it is difficult to observe which attention mask is best.
9 Discussion
Thresholding is an indispensable step in most salient object detection methods. Here, we have presented a novel taxonomy of thresholding methods employed in salient object detection. Few SOD methods output only saliency map without much attention on finding the binary attention mask. Some SOD methods have employed fixed thresholding which do not depend on any image characteristic while some methods have employed all the possible thresholds in a range with suitable steps. The most popular thresholding methods are the fixed range thresholding and the global adaptive thresholding method proposed by Achanta et al. [3]. More than half the research works have employed either of these thresholding methods. Some researchers have modified the multiplying factor to the average saliency value obtained using their proposed methods. Some research works have employed more than one thresholding method for analyzing the performance of their proposed method. The major advantage of global thresholding methods is that these methods are simple to implement and used by most of the researchers in this field. In addition, ROC curve and PR curve can only be drawn with the help of global thresholding which helps in comparing various salient object detection methods. Local thresholding methods have also been used in few research works. However, these methods are scarcely used in literature due to several disadvantages. Hybrid thresholding methods have more complexity than global and local methods rendering their limited use. For automatic salient object detection, thresholding must be fully automatic. There are some methods such as Semi-adaptive thresholding and hybrid thresholding which require human intervention which prohibit their application if the data to be processed is extremely large.
Here, we have presented various thresholding methods employed for salient object detection in literature. But still, there exist some thresholding methods which have not been explored in salient object detection but are extensively employed in image processing context.. These methods can be useful in salient object detection also and are discussed next:
- (i) Histogram shape based::
-
These thresholding methods employ the histogram characteristics of an image for determining the threshold. If the histogram of an individual image is used, then a global adaptive thresholding method can be developed.
- (ii) Entropy based thresholding::
-
Entropy based thresholding has never been employed in salient object detection. These are local methods which employ entropy measure of a pixel together with its neighborhood for determining the threshold. Thus, with the help of entropy based thresholding methods, local thresholding methods for SOD can be developed.
- (iii) Pixel level thresholding::
-
The methods employed for thresholding in SOD either employ global, local or hybrid methods for thresholding. Pixel level thresholding in another approach which is still unexplored for the domain of salient object detection. If a method based on pixel level thresholding is used, this will pave the way for another approach under local thresholding.
- (iv) Hybrid thresholding::
-
A combination of one or more of the above unexplored thresholding methods as well as the ones discussed in Section 3 can be employed for hybrid thresholding methods in SOD.
Besides above thresholding methods, local thresholding methods have also received less attention in comparison to global and hybrid thresholding methods. Thus, local thresholding methods can be further explored for SOD.
10 Conclusion and future work
Salient object detection has become an active research area in last two decades due to its enormous applications in diverse fields. The main objective of SOD methods is to segment the image into foreground and background. SOD methods usually produce a saliency map which is converted to a binary image by applying some thresholding method. In this paper, we have presented a comprehensive survey of various thresholding methods used in salient object detection and developed a novel taxonomy. The thresholding methods are divided into five different categories (i) No Threshold (ii) Global Thresholding (iii) Local Thresholding (iv) Hybrid Thresholding and (v) Other Thresholding methods. Few SOD methods output only the saliency map and use no thresholding. Global thresholding methods employ only single threshold for the whole image. Global thresholding is further divided into three classes (a) Fixed Thresholding (b) Semi Adaptive and (c) Adaptive Thresholding. Fixed thresholding is image independent while global adaptive thresholding is image dependent. In Semi-adaptive thresholding, one factor is set by user and the other is automatically computed from the saliency map of the image at hand. Local thresholding methods apply threshold on superpixels or blocks in the saliency map. Thus, there are multiple thresholds computed in local thresholding methods. In Hybrid methods, either a combination of local and global thresholding or region growing is employed. In region growing methods, an initial threshold is set either manually or saliency map is directly used as initialization. The methods which do not fit in any of these categories are classified under Other Thresholding methods. Most popular thresholding method in salient object detection is Global thresholding which is employed by more than half of the research works in this field. Local thresholding methods are scarcely used but offer a good direction of research. Afterwards, we have also discussed some of the thresholding methods which are employed in image processing but are not explored in salient object detection. Novel thresholding methods provides a good research direction. We have also briefly presented the existing and proposed thresholding performance criteria. Novel performance criteria for SOD performance evaluation is another research direction.
References
Abak AT, Baris U, Sankur B (1997) The performance evaluation of thresholding algorithms for optical character recognition. In: Proceedings of the fourth international conference on document analysis and recognition, 1997, vol 2, pp 697–700. https://doi.org/10.1109/ICDAR.1997.620597
Achanta R, Estrada F, Wils P, Süsstrunk S (2008) Salient region detection and segmentation. In: Proceedings of the 6th international conference on computer vision systems. Springer-Verlag, ICVS’08, pp 66–75
Achanta R, Sheila S H, Francisco J E, Su̇sstrunk S (2009) Frequency-tuned salient region detection. In: 2009 IEEE computer society conference on computer vision and pattern recognition (CVPR 2009), Miami, pp 1597–1604, DOI https://doi.org/10.1109/CVPRW.2009.5206596, (to appear in print)
Alexe B, Deselaers T, Ferrari V (2010) What is an object?. In: The twenty-third IEEE conference on computer vision and pattern recognition, CVPR 2010. San Francisco, pp 73–80. https://doi.org/10.1109/CVPR.2010.5540226
Arya R, Singh N, Agrawal R K (2015) A novel hybrid approach for salient object detection using local and global saliency in frequency domain. Multimed Tools Appl 1–21. https://doi.org/10.1007/s11042-015-2750-y
Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph 26:3. https://doi.org/10.1145/1276377.1276390
Avraham T, Lindenbaum M (2010) Esaliency (extended saliency): meaningful attention using stochastic image modeling. IEEE Trans Pattern Anal Mach Intell 32(4):693–708. https://doi.org/10.1109/TPAMI.2009.53
Bhanu B (1986) Automatic target recognition: state of the art survey. IEEE Trans Aerosp Electron Syst AES-22(4):364–379. https://doi.org/10.1109/TAES.1986.310772
Borji A (2012) Boosting bottom-up and top-down visual features for saliency estimation. In: 2012 IEEE conference on computer vision and pattern recognition. Providence, pp 438–445, DOI https://doi.org/10.1109/CVPR.2012.6247706, (to appear in print)
Borji A (2015) What is a salient object? A dataset and a baseline model for salient object detection. IEEE Trans Image Processing 24(2):742–756. https://doi.org/10.1109/TIP.2014.2383320
Borji A, Itti L (2012) Exploiting local and global patch rarities for saliency detection. In: 2012 IEEE Conference on computer vision and pattern recognition. Providence, pp 478–485, DOI https://doi.org/10.1109/CVPR.2012.6247711, (to appear in print)
Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans PAMI 35(1):185–207. https://doi.org/10.1109/TPAMI.2012.89
Borji A, Sihite DN, Itti L (2013) What stands out in a scene? A study of human explicit saliency judgment. Vis Res 91:62–77. https://doi.org/10.1016/j.visres.2013.07.016
Borji A, Sihite D N, Itti L (2014) What/where to look next? Modeling top-down visual attention in complex interactive environments. IEEE Trans Syst Man Cybern Syst 44(5):523–538. https://doi.org/10.1109/TSMC.2013.2279715
Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell 26(9):1124 –1137. https://doi.org/10.1109/TPAMI.2004.60
Boykovi Y, Lea G F (2006) Graph cuts and efficient n-d image segmentation. Int J Comput Vis 70(2):109–131. https://doi.org/10.1007/s11263-006-7934-5
Bruce N, Tsotsos J (2006) Saliency based on information maximization. Adv Neural Inf Process Syst 18:155–162
Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698. https://doi.org/10.1109/TPAMI.1986.4767851
Chang KY, Liu TL, Chen HT, Lai SH (2011) Fusing generic objectness and visual saliency for salient object detection. In: Proceedings of the 2011 international conference on computer vision ICCV ’11. IEEE Computer Society, Washington, DC, pp 914–921, DOI https://doi.org/10.1109/ICCV.2011.6126333, (to appear in print)
Chen L Q, Xie X, Fan X, Ma W Y, Zhang H J, H Q Zhou H Q (2003) A visual attention model for adapting images on small displays. Multimed Syst 9(4):353–364. https://doi.org/10.1007/s00530-003-0105-4
Chen T, Lin L, Liu L, Luo X, Li X (2015) DISC: deep image saliency computing via progressive representation learning. CoRR arXiv:1511.04192
Cheng M M, Warrell J, Lin W Y, Zheng S, Vineet V, Crook N (2013) Efficient salient region detection with soft image abstraction. In: IEEE International conference on computer vision, ICCV 2013. Sydney, pp 1529–1536, DOI https://doi.org/10.1109/ICCV.2013.193, (to appear in print)
Cheng M M, Mitra N J, Huang X, Torr P H S, Hu S M (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37 (3):569–582. https://doi.org/10.1109/TPAMI.2014.2345401
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1023/A:1022627411411
Deravi F, Pal S K (1983) Grey level thresholding using second-order statistics. Pattern Recogn Lett 1(5–6):417–422. https://doi.org/10.1016/0167-8655(83)90080-6
Dong L, Lin W, Fang Y, Wu S, Seah HS (2014) Saliency detection in computer rendered images based on object-level contrast. J Vis Commun Image Represent 25(3):525–533. https://doi.org/10.1016/j.jvcir.2013.11.009
Du S, Chen S (2014) Salient object detection via random forest. IEEE Signal Process Lett 21(1):51–54. https://doi.org/10.1109/LSP.2013.2290547
Fan Q, Qi C (2014) Two-stage salient region detection by exploiting multiple priors. J Vis Commun Image Represent 25(8):1823–1834. https://doi.org/10.1016/j.jvcir.2014.09.003
Fan Q, Qi C (2016) Saliency detection based on global and local short-term sparse representation. Neurocomput 175(PA):81–89. https://doi.org/10.1016/j.neucom.2015.10.030
Fareed M M S, Ahmed G, Chun Q (2015) Salient region detection through sparse reconstruction and graph-based ranking. J Vis Comun Image Represent 32(C):144–155. https://doi.org/10.1016/j.jvcir.2015.08.002
Fekete G, Eklundh JO, Rosenfeld A (1981) Relaxation: evaluation and applications. IEEE Trans Pattern Anal Mach Intell 3(4):459–469. https://doi.org/10.1109/TPAMI.1981.4767131
Fernandez X (2000) Implicit model-oriented optimal thresholding using the komolgorov-smirnov similarity measure. In: Proceedings 15th international conference on pattern recognition, 2000, vol 1, pp 466–469. https://doi.org/10.1109/ICPR.2000.905377
Frintrop S, García GM, Cremers AB (2014) A cognitive approach for object discovery. In: 22nd international conference on pattern recognition, ICPR 2014. Stockholm, pp 2329–2334. https://doi.org/10.1109/ICPR.2014.404 https://doi.org/10.1109/ICPR.2014.404
Fu K, Gong C, Yang J, Zhou Y, Gu I Y H (2013) Superpixel based color contrast and color distribution driven salient object detection. Signal Process Image Commun 28(10):1448–1463. https://doi.org/10.1016/j.image.2013.07.005
Fu K, Gong C, Gu I Y H, Yang J, He X (2014) Spectral salient object detection. In: IEEE international conference on multimedia and expo, ICME 2014. Chengdu, pp 1–6. https://doi.org/10.1109/ICME.2014.6890142
Fu K, Gong C, Gu I Y H, Yang J (2015) Normalized cut-based saliency detection by adaptive multi-level region merging. IEEE Trans Image Process 24(12):5671–5683. https://doi.org/10.1109/TIP.2015.2485782
Gao HY, Lam KM (2014) Saliency detection based on adaptive dog and distance transform. In: 2014 IEEE international symposium on circuits and systems (ISCAS), pp 534–537. https://doi.org/10.1109/ISCAS.2014.6865190
Gao HY, Lam KM (2014) Salient object detection using octonion with bayesian inference. In: 2014 IEEE international conference on image processing (ICIP), pp 3292–3296. https://doi.org/10.1109/ICIP.2014.7025666
Gao D, Vasconcelos N (2004) Discriminant saliency for visual recognition from cluttered scenes. Adv Neural Inf Process Syst 17:481–488. [Neural Information Processing Systems, NIPS 2004, December 13-18, 2004, Vancouver, British Columbia Canada]
Goferman S, Manor L Z, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926. https://doi.org/10.1109/TPAMI.2011.272
Goldberg C, Chen T, Zhang F L, Shamir A, Hu S M (2012) Data-driven object manipulation in images. Comput Graph Forum 31(2pt1):265–274. https://doi.org/10.1111/j.1467-8659.2012.03005.x
Guo C, Zhang L (2010) A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans Image Process 19(1):185–198. https://doi.org/10.1109/TIP.2009.2030969
Guo C, Ma Q, Zhang L (2008) Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: IEEE Conference on computer vision and pattern recognition, 2008. CVPR 2008, pp 1–8, DOI https://doi.org/10.1109/CVPR.2008.4587715, (to appear in print)
Guo M, Zhao Y, Zhang C, Chen Z (2014) Fast object detection based on selective visual attention. Neurocomputing 144:184–197. https://doi.org/10.1016/j.neucom.2014.04.054
Harel J, Koch C, Perona P (2006) Graph-based visual saliency. In: Proceedings of the twentieth annual conference on neural information processing systems advances in neural information processing systems, vol 19. Vancouver, pp 545–552
He S L, Lau R W H, Liu W H, YQZ (2015) Supercnn: a superpixelwise convolutional neural network for salient object detection. Int J Comput Vis 115 (3):330–344. https://doi.org/10.1007/s11263-015-0822-0
Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: IEEE Conference on computer vision and pattern recognition (CVPR07). IEEE Computer Society, pp 1–8. https://doi.org/10.1109/CVPR.2007.383267
Hou X, Zhang L (2009) Dynamic visual attention: searching for coding length increments. In: Advances in neural information processing systems, vol 21. Curran Associates Inc., pp 681–688
Hu X, Shen J, Shan J, Pan L (2013) Local edge distributions for detection of salient structure textures and objects. IEEE Geosci Remote Sensing Lett 10(3):466–470. https://doi.org/10.1109/LGRS.2012.2210188
Itti L (2004) Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans Image Process 13(10):1304–1318. https://doi.org/10.1109/TIP.2004.834657
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259. https://doi.org/10.1109/34.730558
Jawahar C, Biswas P, Ray A (1997) Investigations on fuzzy thresholding based on fuzzy clustering. Pattern Recog 30(10):1605–1613. https://doi.org/10.1016/S0031-3203(97)00004-6
Ji Q G, Fang Z D, Xie Z H, Lu Z M (2013) Video abstraction based on the visual attention model and online clustering. Image Commun 28(3):241–253. https://doi.org/10.1016/j.image.2012.11.008
Jia Y, Han M (2013) Category-independent object-level saliency detection. In: IEEE International conference on computer vision, ICCV 2013. Sydney, pp 1761–1768. https://doi.org/10.1109/ICCV.2013.221
Jia C, Qi J, Li X, Lu H (2016) Saliency detection via a unified generative and discriminative model. Neurocomputing 173(Part 2):406–417. https://doi.org/10.1016/j.neucom.2015.03.122
Jian M, Lam K M, Dong J, Shen L (2015) Visual-patch-attention-aware saliency detection. IEEE Trans Cybern 45(8):1575–1586. https://doi.org/10.1109/TCYB.2014.2356200
Jiang Z, Davis L S (2013) Submodular salient region detection. In: 2013 IEEE conference on computer vision and pattern recognition. Portland, pp 2043–2050, DOI https://doi.org/10.1109/CVPR.2013.266, (to appear in print)
Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR), pp 2083–2090. https://doi.org/10.1109/CVPR.2013.271
Jin Z, Han J, Zhang Y, Bai L (2015) Saliency model based on a discrete centre-surround. Electron. Lett. 51(8):626–628. https://doi.org/10.1049/el.2014.4316
Johannsen G, Bille J (1982) A threshold selection method using information measures. In: Proceedings of the 6th international conference on pattern recognition, pp 140–143
Ju R, Liu Y, Ren T, Ge L, Wu G (2015) Depth-aware salient object detection using anisotropic center-surround difference. Signal Process Image Commun 38:115–126. https://doi.org/10.1016/j.image.2015.07.002 Recent Advances in Saliency Models, Applications and Evaluations
Judd T, Ehinger K, Durand F, Torralba A (2009) Learning to predict where humans look. In: IEEE international conference on computer vision (ICCV), pp 2106–2113. https://doi.org/10.1109/ICCV.2009.5459462
Kamel M, Zhao A (1993) Extraction of binary character/graphics images from grayscale document images. CVGIP: Graph Models Image Process 55(3):203–217. https://doi.org/10.1006/cgip.1993.1015
Kannan R, Ghinea G, Swaminathan S (2015) Salient region detection using patch level and region level image abstractions. IEEE Signal Process Lett 22(6):686–690. https://doi.org/10.1109/LSP.2014.2366192
Kapur J, Sahoo P, Wong A (1985) A new method for gray-level picture thresholding using the entropy of the histogram. Comput Vis Graph Image Process 29(3):273–285. https://doi.org/10.1016/0734-189X(85)90125-2
Karpathy A, Miller S D, Li F F (2013) Object discovery in 3d scenes via shape analysis. In: 2013 IEEE International conference on robotics and automation. Karlsruhe, pp 2088–2095, https://doi.org/10.1109/ICRA.2013.6630857, (to appear in print)
Khuwuthyakorn P, Kelly A R, Zhou J (2010) Computer vision – ECCV 2010: 11th European conference on computer vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part II. Chap object of interest detection by saliency learning. Springer, Berlin, pp 636–649, DOI https://doi.org/10.1007/978-3-642-15552-9-46, (to appear in print)
Kienzle W, Franz M, Schölkopf B, Wichmann F (2009) Center-surround patterns emerge as optimal predictors for human saccade targets. J Vis 9(5:7):1–15
Kim J, Lee H, Kim J (2013) A novel method for salient object detection via compactness measurement. In: IEEE International conference on image processing, ICIP 2013. Melbourne, pp 3426–3430, DOI https://doi.org/10.1109/ICIP.2013.6738707, (to appear in print)
Kim J, Han D, Tai YW, Kim J (2014) Salient region detection via high-dimensional color transform. In: 2014 IEEE Conference on computer vision and pattern recognition, pp 883–890. https://doi.org/10.1109/CVPR.2014.118
Kirby R L, Rosenfeld A (1979) A note on the use of (gray level, local average gray level) space as an aid in threshold selection. IEEE Trans Syst Man Cybern 9(12):860–864. https://doi.org/10.1109/TSMC.1979.4310138 https://doi.org/10.1109/TSMC.1979.4310138
Kittler J, Illingworth J (1986) Minimum error thresholding. Pattern Recogn 19(1):41–47. https://doi.org/10.1016/0031-3203(86)90030-0
Klein D A, Frintrop S (2011) Center-surround divergence of feature statistics for salient object detection. In: IEEE International conference on computer vision, ICCV 2011. Barcelona, pp 2214–2219, DOI https://doi.org/10.1109/ICCV.2011.6126499, (to appear in print)
Koch C, Ullman S (1985) Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol 4:219–227. https://doi.org/10.1007/978-94-009-3833-5-5
Koch C, Ullman S (1987) Matters of intelligence: conceptual structures in cognitive neuroscience. Chap shifts in selective visual attention: towards the underlying neural circuitry. Springer, Netherlands, pp 115–141, DOI https://doi.org/10.1007/978-94-009-3833-5-5, (to appear in print)
Kong L, Duan L, Yang W, Dou Y (2015) Salient region detection: an integration approach based on image pyramid and region property. IET Comput Vis 9(1):85–97. https://doi.org/10.1049/iet-cvi.2013.0285
Ksantini R, Boufama B, Memar S (2013) A new efficient active contour model without local initializations for salient object detection. EURASIP J Image Video Process 2013:40. https://doi.org/10.1186/1687-5281-2013-40
Kumar N, Singh M, Govil M C, Pilli E S, Jaiswal A (2016) Salient object detection in noisy images. In: Proceedings of the 29th Canadian conference on artificial intelligence on advances in artificial intelligence - 9673. Springer-Verlag New York, Inc., New York, pp 109–114. https://doi.org/10.1007/978-3-319-34111-8-15
Kwak S Y, Ko B, Byun H (2004) Automatic salient-object extraction using the contrast map and salient points. In: 5th Pacific Rim conference on multimedia. Tokyo, pp 138–145. https://doi.org/10.1007/978-3-540-30542-2-18
Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the eighteenth international conference on machine learning ICML ’01. Morgan Kaufmann Publishers Inc., San Francisco, pp 282–289
Lang C, Feng J, Feng S, Wang J, Yan S (2016) Dual low-rank pursuit: learning salient features for saliency detection. IEEE Trans Neural Netw Learn Syst 27(6):1190–1200. https://doi.org/10.1109/TNNLS.2015.2513393
Leitner J, Harding S, Chandrashekhariah P, Frank M, Förster A, Triesch J, Schmidhuber J (2013) Learning visual object detection and localisation using icvision. Biolog Insp Cogn Architect 5(0):29–41. https://doi.org/10.1016/j.bica.2013.05.009
Leung C, Lam F (1996) Performance analysis for a class of iterative image thresholding algorithms. Pattern Recogn 29(9):1523–1530. https://doi.org/10.1016/0031-3203(96)00009-X
Leung C, Lam F (1998) Maximum segmented image information thresholding. Graph Models Image Process 60(1):57–76. https://doi.org/10.1006/gmip.1997.0455
Li Z, Chen J (2008) On semantic object detection with salient feature. In: Advances in visual computing, 4th international symposium, ISVC December 1-3, 2008. Proceedings, Part II, pp. 782–791. https://doi.org/10.1007/978-3-540-89646-3-77
Li H, Ngan K N (2011) A co-saliency model of image pairs. IEEE Trans Image Process 20(12):3365–3375. https://doi.org/10.1109/TIP.2011.2156803
Li C, Tam P (1998) An iterative algorithm for minimum cross entropy thresholding. Pattern Recogn Lett 19 (8):771–776. https://doi.org/10.1016/S0167-8655(98)00057-9
Li X, Li Y, Shen C, Dick A, Hengel AVD (2013) Contextual hypergraph modeling for salient object detection. In: Proceedings of the 2013 IEEE international conference on computer vision ICCV ’13. Washington, DC, pp 3328–3335, DOI https://doi.org/10.1109/ICCV.2013.413, (to appear in print)
Li Y, Fu K, Zhou L, Qiao Y, Yang J, Li B (2014) Saliency detection based on extended boundary prior with foci of attention. In: 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2798–2802. https://doi.org/10.1109/ICASSP.2014.6854110
Li C, Hu Z, Xiao L, Pan Z (2015) Saliency detection via low-rank reconstruction from global to local. In: Chinese automation congress (CAC) 2015, pp 669–673. https://doi.org/10.1109/CAC.2015.7382582
Li J, Meng F, Zhang Y (2015) Saliency detection using a background probability model. In: 2015 IEEE international conference on image processing (ICIP), pp 2189–2193. https://doi.org/10.1109/ICIP.2015.7351189
Li S, Lu H, Lin Z L, Shen X, Price B L (2015) Adaptive metric learning for saliency detection. IEEE Trans Image Process 24(11):3321–3331
Liang Z, Chi Z, Fu H, Feng D D (2012) Salient object detection using content-sensitive hypergraph representation and partitioning. Pattern Recogn 45 (11):3886–3901. https://doi.org/10.1016/j.patcog.2012.04.017
Lin M, Zhang C, Chen Z (2015) Global feature integration based salient region detection. Neurocomputing 159:1–8. https://doi.org/10.1016/j.neucom.2015.02.050
Lin M, Zhang C, Chen Z (2016) Predicting salient object via multi-level features. Neurocomputing 205:301–310. https://doi.org/10.1016/j.neucom.2016.04.036. http://www.sciencedirect.com/science/article/pii/S0925231216303010
LItti, Baldi P (2009) Bayesian surprise attracts human attention. Vis Res 49(10):1295–1306. https://doi.org/10.1016/j.visres.2008.09.007. Visual attention: psychophysics, electrophysiology and neuroimaging
Liu F, Gleicher M (2006) Region enhanced scale-invariant saliency detection. In: Proceedings of the 2006 IEEE international conference on multimedia and expo, ICME 2006, pp 1477–1480 https://doi.org/10.1109/ICME.2006.262821
Liu H, Heynderickx I (2009) Studying the added value of visual attention in objective image quality metrics based on eye movement data. In: 2009 16th IEEE international conference on image processing (ICIP), pp 3097–3100. https://doi.org/10.1109/ICIP.2009.5414466
Liu J, Wang S (2015) Salient region detection via simple local and global contrast representation. Neurocomputing 147:435–443. https://doi.org/10.1016/j.neucom.2014.06.041. Advances in Self-Organizing Maps Subtitle of the special issue: Selected Papers from the Workshop on Self-Organizing Maps 2012 (WSOM 2012)
Liu T, Sun J, Zheng N, Tang X, Shum HY (2007) Learning to detect a salient object. In: IEEE computer society conference on computer vision and pattern recognition, pp 1–8. https://doi.org/10.1109/CVPR.2007.383047 https://doi.org/10.1109/CVPR.2007.383047
Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum H Y (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33 (2):353–367. https://doi.org/10.1109/TPAMI.2010.70
Liu Q, Han T, Sun Y, Chu Z, Shen Z B (2012) A two step salient objects extraction framework based on image segmentation and saliency detection. Multimed Tools Appl 67(1):231–247. https://doi.org/10.1007/s11042-012-1077-1
Liu Z, Zou W, Meur O L (2014) Saliency tree: a novel saliency detection framework. IEEE Trans Image Process 23(5):1937–1952. https://doi.org/10.1109/TIP.2014.2307434
Liu R, Cao J, Lin Z, Shan S (2014) Adaptive partial differential equation learning for visual saliency detection. In: 2014 IEEE conference on computer vision and pattern recognition, pp 3866–3873. https://doi.org/10.1109/CVPR.2014.494
Liu Y, Cai Q, Zhu X, Cao J, Li H (2015) Saliency detection using two-stage scoring. In: 2015 IEEE International conference on image processing (ICIP), pp 4062–4066. https://doi.org/10.1109/ICIP.2015.7351569 https://doi.org/10.1109/ICIP.2015.7351569
Liu Z, Gu G, Chen C, Cui D, Lin C (2016) Background priors based saliency object detection. In: 2016 Asia-Pacific signal and information processing association annual summit and conference (APSIPA), pp 1–4. https://doi.org/10.1109/APSIPA.2016.7820744
Lu S, Mahadevan V, Vasconcelos N (2014) Learning optimal seeds for diffusion-based salient object detection. In: 2014 IEEE conference on computer vision and pattern recognition, pp 2790–2797. https://doi.org/10.1109/CVPR.2014.357
Lu H, Li X, Zhang L, Ruan X, Yang M H (2016) Dense and sparse reconstruction error based saliency descriptor. IEEE Trans Image Process 25 (4):1592–1603. https://doi.org/10.1109/TIP.2016.2524198
Luo Y, Yuan J, Xue P, Tian Q (2011) Saliency density maximization for efficient visual objects discovery. IEEE Trans Circ Syst Vid Techn 21(12):1822–1834. https://doi.org/10.1109/TCSVT.2011.2147230
Luo W, Li H, Liu G, Ngan K N (2012) Global salient information maximization for saliency detection. Sig Proc Image Comm 27(3):238–248. https://doi.org/10.1016/j.image.2011.10.004
Ma Y F, Zhang H J (2003) Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of the eleventh ACM international conference on multimedia, pp 374–381. https://doi.org/10.1145/957013.957094
Ma Y F, Hua X S, Lu L, Zhang H J (2005) A generic framework of user attention model and its application in video summarization. IEEE Trans Multimed 7(5):907–919. https://doi.org/10.1109/TMM.2005.854410
Ma X, Xie X, Lam K M, Hu J, Zhong Y (2015) Saliency detection based on singular value decomposition. J Vis Comun Image Represent 32(C):95–106. https://doi.org/10.1016/j.jvcir.2015.08.003
Mahmoudi L, Zaart AE (2012) A survey of entropy image thresholding techniques. In: 2012 2nd international conference on advances in computational tools for engineering applications (ACTEA), pp 204–209. https://doi.org/10.1109/ICTEA.2012.6462867
Manipoonchelvi P, Muneeswaran K (2014) Region-based saliency detection. IET Image Process 8(9):519–527. https://doi.org/10.1049/iet-ipr.2013.0434
Manke R, Jalal A S (2015) Poisson-distribution-based approach for salient region detection. Electron Lett 51(1):37–38. https://doi.org/10.1049/el.2014.3334
Marchesotti L, Cifarelli C, Csurka G (2009) A framework for visual saliency detection with applications to image thumbnailing. In: IEEE 12th international conference on computer vision, ICCV 2009. Kyoto, pp 2232-2239. https://doi.org/10.1109/ICCV.2009.5459467
Margolin R, Tal A, Zelnik-Manor L (2013) What makes a patch distinct? In: 2013 IEEE conference on computer vision and pattern recognition (CVPR), pp 1139–1146. https://doi.org/10.1109/CVPR.2013.151
Martín RV, Marfil R, Nu̇ṅez P, Bandera A, Hernȧndez FS (2009) A novel approach for salient image regions detection and description. Pattern Recogn Lett 30(16):1464–1476. https://doi.org/10.1016/j.patrec.2009.08.003
Mehrani P, Veksler O (2010) Saliency segmentation based on learning and graph cut refinement. In: Proceedings of the British machine vision conference. BMVA Press, pp 110.1–110.12. https://doi.org/10.5244/C.24.110
Muratov O, Boato G, Natale FGBD (2013) Salient object detection using scene layout estimation. In: 2013 IEEE 15th international workshop on multimedia signal processing (MMSP), pp 390–395. https://doi.org/10.1109/MMSP.2013.6659320
Murthy C A, Pasl S K (1990) Fuzzy thresholding: mathematical framework, bound functions and weighted moving average technique. Pattern Recogn Lett 11 (3):197–206. https://doi.org/10.1016/0167-8655(90)90006-N
Nakagawa Y, Rosenfeld A (1979) Some experiments on variable thresholding. Pattern Recogn 11(3):191–204. https://doi.org/10.1016/0031-3203(79)90006-2
Naqvi S S, Browne W N, Hollitt C (2016) Salient object detection via spectral matting. Pattern Recogn 51(C):209–224. https://doi.org/10.1016/j.patcog.2015.09.026
Navalpakkam V, Itti L (2005) Modeling the influence of task on attention. Vis Res 45(2):205–231. https://doi.org/10.1016/j.visres.2004.07.042
Niblack W (1985) An introduction to digital image processing. Strandberg Publishing Company Birkeroed, Denmark
Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66. https://doi.org/10.1109/TSMC.1979.4310076
Pal S, King R, Hashim A (1983) Automatic grey level thresholding through index of fuzziness and entropy. Pattern Recogn Lett 1(3):141–146. https://doi.org/10.1016/0167-8655(83)90053-3
Palumbo P W, Swaminathan P, Srihari S N (1986) Document image binarization: evaluation of algorithms. Proc SPIE Appl Digit Image Process 0697:278–286. https://doi.org/10.1117/12.976229
Peng H, Li B, Ling H, Hu W, Xiong W, Maybank S J (2017) Salient object detection via structured matrix decomposition. IEEE Trans Pattern Anal Mach Intell 39(4):818–832. https://doi.org/10.1109/TPAMI.2016.2562626
Perazzi F, Krȧhenbu̇hl P, Pritch Y, Hornung A (2012) Saliency filters: contrast based filtering for salient region detection. In: 2012 IEEE Conference on computer vision and pattern recognition. Providence, pp 733–740, DOI https://doi.org/10.1109/CVPR.2012.6247743, (to appear in print)
Peters RJ, Itti L (2007) Beyond bottom-up: incorporating task-dependent influences into a computational model of spatial attention. In: 2007 IEEE conference on computer vision and pattern recognition, pp 1–8. https://doi.org/10.1109/CVPR.2007.383337
Pun T (1980) A new method for grey-level picture thresholding using the entropy of the histogram. Signal Process 2(3):223–237. https://doi.org/10.1016/0165-1684(80)90020-1
Pun T (1981) Entropic thresholding, a new approach. Comput Graph Image Process 16(3):210–239. https://doi.org/10.1016/0146-664X(81)90038-1
Qin C, Zhang G, Zhou Y, Tao W, Cao Z (2014) Integration of the saliency-based seed extraction and random walks for image segmentation. Neurocomput 129:378–391. https://doi.org/10.1016/j.neucom.2013.09.021
Qi S, Yu J G, Ma J, Li Y, Tian J (2015) Salient object detection via contrast information and object vision organization cues. Neurocomput 167(C):390–405. https://doi.org/10.1016/j.neucom.2015.04.055
Qi W, Han J, Zhang Y, Bai L (2015) Saliency detection via boolean and foreground in a dynamic Bayesian framework. Vis Comput 1–12. https://doi.org/10.1007/s00371-015-1176-x
Qi W, Cheng M M, Borji A, Lu H, Bai L F (2016) Saliencyrank: two-stage manifold ranking for salient object detection. Comput Vis Media 1(4):309–320. https://doi.org/10.1007/s41095-015-0028-y
Qin Y, Lu H, Xu Y, Wang H (2015) Saliency detection via cellular automata. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 110–119. https://doi.org/10.1109/CVPR.2015.7298606
Rahtu E, Kannala J, Salo M, Heikkilä J (2010) Segmenting salient objects from images and videos. In: Proceedings of the 11th European conference on computer vision: Part V, ECCV’10. Berlin, Heidelberg, pp 366–379
Ramar K, Arumugam S, Sivanandam S, Ganesan L, Manimegalai D (2000) Quantitative fuzzy measures for threshold selection. Pattern Recogn Lett 21(1):1–7. https://doi.org/10.1016/S0167-8655(99)00120-8
Ramström O, Christensen HI (2002) Visual attention using game theory. In: Biologically motivated computer vision second international workshop, BMCV 2002. Tübingen, Germany, November 22-24, 2002, Proceedings, pp 462–471. https://doi.org/10.1007/3-540-36181-2-46
Rao RP, Gregory JZ, Hayhoe MM, Dana HB (2002) Eye movements in iconic visual search. Vis Res 42(11):1447–1463. https://doi.org/10.1016/S0042-6989(02)00040-8
Ren Y F, Mu Z C (2014) Salient object detection based on global contrast on texture and color. In: 2014 international conference on machine learning and cybernetics, vol 1, pp 7–12. https://doi.org/10.1109/ICMLC.2014.7009083
Ren Z, Gao S, Chia L T, Tsang I W H (2014) Region-based saliency detection and its application in object recognition. IEEE Trans Circ Syst Vid Technol 24(5):769–779. https://doi.org/10.1109/TCSVT.2013.2280096
Rijsbergen C J V (1979) Information retrieval, 2nd edn. Butterworth-Heinemann, Newton
Rodhetbhai W, Lewis PH (2007) Salient region filtering for background subtraction. In: Proceedings of the 9th international conference on advances in visual information systems, VISUAL’07, pp 126–135
Rosenfeld A, Torre P D L (1983) Histogram concavity analysis as an aid in threshold selection. IEEE Trans Syst Man Cybern SMC-13(2):231–235. https://doi.org/10.1109/TSMC.1983.6313118
Rosin P L (2009) A simple method for detecting salient regions. Pattern Recogn 42(11):2363–2371. https://doi.org/10.1016/j.patcog.2009.04.021
Rother C, Kolmogorov V, Blake A (2004) “grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309–314. https://doi.org/10.1145/1015706.1015720
Roy S, Das S (2013) Spatial variance of color and boundary statistics for salient object detection. In: 2013 Fourth national conference on computer vision, pattern recognition, image processing and graphics (NCVPRIPG), pp 1–4. https://doi.org/10.1109/NCVPRIPG.2013.6776270
Sahoo P, Wilkins C, Yeager J (1997) Threshold selection using renyi’s entropy. Pattern Recogn 30(1):71–84. https://doi.org/10.1016/S0031-3203(96)00065-9
Sauvola J, Pietikäinen M (2000) Adaptive document image binarization. Pattern Recogn 33:225– 236
Scharfenberger C, Wong A, Fergani K, Zelek JS, Clausi DA (2013) Statistical textural distinctiveness for salient region detection in natural images. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR), pp 979–986. https://doi.org/10.1109/CVPR.2013.131
Seo Y, Yoo C D (2014) Salient object detection based on sparse representation with image-specific prior. In: The 18th IEEE international symposium on consumer electronics (ISCE 2014), pp 1–2. https://doi.org/10.1109/ISCE.2014.6884549
Sezan M I (1990) A peak detection algorithm and its application to histogram-based image data reduction. Comput Vis Graph Image Process 49(1):36–51. https://doi.org/10.1016/0734-189X(90)90161-N
Sezgin M, Sankur B (2004) Survey over image thresholding techniques and quantitative performance evaluation. J Electron Imaging 13(1):146–168. https://doi.org/10.1117/1.1631315
Sezgin M, Taşaltin R (2000) A new dichotomization technique to multilevel thresholding devoted to inspection applications. Pattern Recogn Lett 21 (2):151–161. https://doi.org/10.1016/S0167-8655(99)00142-7 https://doi.org/10.1016/S0167-8655(99)00142-7
Shao L, Brady M (2006) Invariant salient regions based image retrieval under viewpoint and illumination variations. J Vis Commun Image Represent 17(6):1256–1272. https://doi.org/10.1016/j.jvcir.2006.08.002
Shen X, Wu Y (2012) A unified approach to salient object detection via low rank matrix recovery. In: 2012 IEEE conference on computer vision and pattern recognition. Providence, pp 853–860. https://doi.org/10.1109/CVPR.2012.6247758
Shi J, Yan Q, Xu L, Jia J (2016) Hierarchical image saliency detection on extended cssd. IEEE Trans Pattern Anal Mach Intell 38(4):717–729. https://doi.org/10.1109/TPAMI.2015.2465960
Siagian C, Itti L (2009) Biologically inspired mobile robot vision localization. IEEE Trans Robot 25(4):861–873. https://doi.org/10.1109/TRO.2009.2022424
Singh N, Agrawal R K (2015) Combination of kullback-leibler divergence and manhattan distance measures to detect salient objects. SIViP 9(2):427–435. https://doi.org/10.1007/s11760-013-0457-y
Singh A, Chu CHH, Pratt MA (2014) Multiresolution superpixels for visual saliency detection. In: 2014 IEEE symposium on computational intelligence for multimedia, signal and vision processing (CIMSIVP), pp 1–8. https://doi.org/10.1109/CIMSIVP.2014.7013277
Singh N, Arya R, Agrawal R K (2014) A novel approach to combine features for salient object detection using constrained particle swarm optimization. Pattern Recogn 47(4):1731–1739. https://doi.org/10.1016/j.patcog.2013.11.012
Siva P, Russell C, Xiang T, Agapito L (2013) Looking beyond the image: unsupervised learning for object saliency and detection. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR), pp 3238–3245. https://doi.org/10.1109/CVPR.2013.416
Sugano Y, Matsushita Y, Sato Y (2010) Calibration-free gaze sensing using saliency maps. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR), pp 2667–2674. https://doi.org/10.1109/CVPR.2010.5539984
Sun X, Zhu Z, Liu X, Shang Y, Yu Q (2015) Frequency-spatial domain based salient region detection. Optik - Int J Light Electron Opt 126(9–10):942–949. https://doi.org/10.1016/j.ijleo.2015.03.004
Tang C, Hou C, Wang P, Song Z (2015) Salient object detection using color spatial distribution and minimum spanning tree weight. Multimed Tools Appl 1–16. https://doi.org/10.1007/s11042-015-2622-5
Tang Y, Tong R, Tang M, Zhang Y (2015) Depth incorporating with color improves salient object detection. Vis Comput 32(1):111–121. https://doi.org/10.1007/s00371-014-1059-6
Tong N, Lu H, Zhang L, Ruan X (2014) Saliency detection with multi-scale superpixels. IEEE Signal Process Lett 21(9):1035–1039. https://doi.org/10.1109/LSP.2014.2323407
Tong N, Lu H, Zhang Y, Ruan X (2015) Salient object detection via global and local cues. Pattern Recogn 48(10):3258–3267. https://doi.org/10.1016/j.patcog.2014.12.005
Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12(1):97–136. https://doi.org/10.1016/0010-0285(80)90005-5
Trier O D, Jain A K (1995) Goal-directed evaluation of binarization methods. IEEE Trans Pattern Anal Mach Intell 17(12):1191–1201. https://doi.org/10.1109/34.476511
Tsai W H (1995) Moment-preserving thresholding: a new approach. In: O’Gorman L, Kasturi R (eds) Document image analysis. IEEE Computer Society Press, Los Alamitos, pp 44–60
Valenti R, Sebe N, Gevers T (2009) Image saliency by isocentric curvedness and color. In: ICCV, IEEE computer society, pp 2185–2192. https://doi.org/10.1109/ICCV.2009.5459240
Walther D, Koch C (2006) Modeling attention to salient proto-objects. Neural Netw 19(9):1395–1407. https://doi.org/10.1016/j.neunet.2006.10.001
Wang H B, Lv H (2016) Salient object detection with fixation priori. In: 2016 international conference on machine learning and cybernetics (ICMLC), vol 1, pp 285–289. https://doi.org/10.1109/ICMLC.2016.7860915
Wang Z, Wu X (2016) Salient object detection using biogeography-based optimization to combine features. Appl Intell 1–17. https://doi.org/10.1007/s10489-015-0739-x
Wang W, Cai D, Xu X, Liew AWC (2014) Visual saliency detection based on region descriptors and prior knowledge. Signal Process Image Commun 29(3):424–433. https://doi.org/10.1016/j.image.2014.01.004
Wang H, Zhang P, Liu J (2015) Salient region detection by learning accurate background template. In: The 27th Chinese control and decision conference (2015 CCDC), pp 2519–2524. https://doi.org/10.1109/CCDC.2015.7162345
Wang Z, Jiang P, Wang F, Zhang X (2016) Recurrent double features: recurrent multi-scale deep features and saliency features for salient object detection. Springer International Publishing, Cham, pp 376–386
Weszka J S, Rosenfeld A (1977) Histogram modification for threshold selection. NASA STI/Recon Technical Report N 78
White J M, Rohrer G D (1983) Image thresholding for optical character recognition and other applications requiring character image extraction. IBM J Res Dev 27(4):400–411. https://doi.org/10.1147/rd.274.0400
Wu A Y, Hong T H, Rosenfeld A (1982) Threshold selection using quadtrees. IEEE Trans Pattern Anal Mach Intell PAMI-4(1):90–94. https://doi.org/10.1109/TPAMI.1982.4767203
Xiang D, Wang Z (2016) Salient object detection via saliency bias and diffusion. Multimed Tools Appl 1–20. https://doi.org/10.1007/s11042-016-3310-9
Xie Y, Lu H, Yang M H (2013) Bayesian saliency via low and mid level cues. IEEE Trans Image Process 22(5):1689–1698. https://doi.org/10.1109/TIP.2012.2216276
Xu K, Chen X (2013) A multi-stage area saliency detection model. In: 2013 4th IEEE international conference on software engineering and service science (ICSESS), pp 865–869. https://doi.org/10.1109/ICSESS.2013.6615442
Xu L, Zeng L, Duan H, Sowah N L (2014) Saliency detection in complex scenes. EURASIP J Image Vid Process 2014(1):1–13. https://doi.org/10.1186/1687-5281-2014-31
Xu X, Mu N, Zhang H, Fu X (2015) Salient object detection from distinctive features in low contrast images. In: 2015 IEEE international conference on image processing (ICIP), pp 3126–3130. https://doi.org/10.1109/ICIP.2015.7351379
Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR), pp 3166–3173. https://doi.org/10.1109/CVPR.2013.407
Yan X, Wang Y, Jiang M, Wang J (2014) Salient region detection via color spatial distribution determined global contrasts. In: 2014 IEEE international conference on image processing (ICIP), pp 1170–1174. https://doi.org/10.1109/ICIP.2014.7025233
Yang X, Qian X, Mei T (2015) Learning salient visual word for scalable mobile image retrieval. Pattern Recogn 48(10):3093–3101. https://doi.org/10.1016/j.patcog.2014.12.017. Discriminative feature learning from big data for visual recognition
Yasuda Y, Dubois M, Huang T S (1980) Data compression for check processing machines. Proc IEEE 68(7):874–885. https://doi.org/10.1109/PROC.1980.11753
Yeh M C, Hsu C F, Lu C J (2014) Fast salient object detection through efficient subwindow search. Pattern Recogn Lett 46:60–66. https://doi.org/10.1016/j.patrec.2014.05.006
Yu H, Li J, Tian Y, Huang T (2010) Automatic interesting object extraction from images using complementary saliency maps. In: Proceedings of the 18th ACM international conference on multimedia MM ’10, New York, pp 891–894, DOI https://doi.org/10.1145/1873951.1874105, (to appear in print)
Zhang Y J (1996) A survey on evaluation methods for image segmentation. Pattern Recogn 29(8):1335–1346. https://doi.org/10.1016/0031-3203(95)00169-7
Zhang D, Liu C (2014) A salient object detection framework beyond top-down and bottom-up mechanism. Biologically Insp Cogn Architect 9:1–8. https://doi.org/10.1016/j.bica.2014.06.005. Neural-symbolic networks for cognitive capacities
Zhang L, Yuan X (2015) Salient object detection with higher order potentials and learning affinity. IEEE Signal Processing Lett 22(9):1396–1399. https://doi.org/10.1109/LSP.2014.2377216
Zhang L, Tong M H, Marks T K, Cottrell G W (2008) SUN: a Bayesian framework for saliency using natural statistics. J Vis 8:1–20. https://doi.org/10.1167/8.7.32.Introduction
Zhang L, Gu Z, Li H (2013) Sdsp: a novel saliency detection method by combining simple priors. In: 2013 IEEE international conference on image processing, pp 171–175. https://doi.org/10.1109/ICIP.2013.6738036
Zhang J, Ehinger KA, Ding J, Yang J (2014) A prior-based graph for salient object detection. In: 2014 IEEE international conference on image processing (ICIP), pp 1175–1178. https://doi.org/10.1109/ICIP.2014.7025234 https://doi.org/10.1109/ICIP.2014.7025234
Zhang Y Y, Liu X Y, Wang H J (2014) Saliency detection via two-directional 2dpca analysis of image patches. Optik - Int J Light Electron Opt 125(24):7222–7226. https://doi.org/10.1016/j.ijleo.2014.07.132
Zhang M M, Li Z M, Bai H H, Sun Y (2014) Robust image salient regional extraction and matching based on dogss-msers. Optik - Int J Light Electron Opt 125(3):1469–1473. https://doi.org/10.1016/j.ijleo.2013.09.007
Zhang W, Xiong Q, Shi W, Chen S (2015) Region saliency detection via multi-feature on absorbing Markov chain. Vis Comput 32(3):275–287. https://doi.org/10.1007/s00371-015-1065-3
Zhang Q, Lin J, Li X (2016) Salient object detection via structure extraction and region contrast. J Inf Sci Eng 32:1435–1454
Zhang J, Sclaroff S, Lin Z, Shen X, Price B, Mech R (2016) Unconstrained salient object detection via proposal subset optimization. In: The IEEE conference on computer vision and pattern recognition (CVPR), pp 5733–5742
Zhang J, Ehinger KA, Wei H, Zhang K, Yang J (2017) A novel graph-based optimization framework for salient object detection. Pattern Recogn 64:39–50. https://doi.org/10.1016/j.patcog.2016.10.025
Zhang Q, Lin J, Tao Y, Li W, Shi Y (2017) Salient object detection via color and texture cues. Neurocomputing 243:35–48. https://doi.org/10.1016/j.neucom.2017.02.064
Zhao H, Chen J, Han Y, Cao X (2014) Image aesthetics enhancement using composition-based saliency detection. Multimed Syst 21(2):159–168. https://doi.org/10.1007/s00530-014-0373-1
Zhao R, Ouyang W, Li H, Wang X (2015) Saliency detection by multi-context deep learning. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 1265–1274. https://doi.org/10.1109/CVPR.2015.7298731
Zhou L, Li YJ, Song YP, Qiao Y, Yang J (2014) Saliency driven clustering for salient object detection. In: 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 5372–5376. https://doi.org/10.1109/ICASSP.2014.6854629
Zhou L, Yang Z, Chang G (2015) Salient region detection based on compactness with manifold ranking. In: 2015 5th international conference on information science and technology (ICIST), pp 108–112. https://doi.org/10.1109/ICIST.2015.7288950
Zhou L, Yang Z, Yuan Q, Zhou Z, Hu D (2015) Salient region detection via integrating diffusion-based compactness and local contrast. IEEE Trans Image Process 24(11):3308–3320. https://doi.org/10.1109/TIP.2015.2438546
Zhou Q, Li N, Chen J, Cai S, Latecki LJ (2015) Salient object detection via background contrast. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1463–1467. https://doi.org/10.1109/ICASSP.2015.7178213
Zhu L, Klein D A, Frintrop S, Cao Z, Cremers A B (2014) A multisize superpixel approach for salient object detection based on multivariate normal distribution estimation. IEEE Trans Image Process 23(12):5094–5107. https://doi.org/10.1109/TIP.2014.2361024
Zou W, Kpalma K, Liu Z, Ronsin J (2013) Segmentation driven low-rank matrix recovery for saliency detection. In: British Machine vision conference, BMVC 2013. Bristol, pp 1–13. https://doi.org/10.5244/C.27.78
Zou B, Liu Q, Chen Z, Liu S, Zhang X (2015) Saliency detection using boundary information. Multimed Syst 22 (2):245–253. https://doi.org/10.1007/s00530-014-0449-y
Zou W, Liu Z, Kpalma K, Ronsin J, Zhao Y, Komodakis N (2015) Unsupervised joint salient region detection and object segmentation. IEEE Trans Image Process 24(11):3858–3873. https://doi.org/10.1109/TIP.2015.2456497
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kumar, N. Thresholding in salient object detection: a survey. Multimed Tools Appl 77, 19139–19170 (2018). https://doi.org/10.1007/s11042-017-5329-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-5329-y