1 Introduction

With the integration of new multimedia capabilities into wireless sensor devices, Wireless Multimedia Sensor Networks (WMSN) has gained more attention of the research community [1, 2]. Multimedia enabled sensor nodes can collect and transmit multimedia data (audio, image, video) from lossy environment. These networks provide a wide range of applications from environmental applications, such as animal habitats, building monitoring, to surveillance systems such as locating missing people, border monitoring and identifying criminals. Further examples of practical applications for WMSN can be found in [1, 3]. Efficiently gathering and transmitting data in order to satisfy the application layer requirements in WMSN is imperative. Many studies related to multimedia transmission in WMSN are presented [1, 4, 5]. Due to the rigorous reliability demand of multimedia transmission over wireless media, application-specific requirements are also addressed in [6–8].

During transmission of an image over WMSN, image packets may be subject to losses due to channel impairments, node failures etc. This dynamic nature of the wireless communication causes packet losses, which noticeably affect the perceptual quality of the application. Few studies consider the impact of the packet loss on multimedia quality [5, 9–12]. Various different communication protocols are studied to increase the performance of the multimedia transmission in WMSN [13–16]. However, few of them have considered the impact of the used image quality evaluation metric on obtained communication performance [5, 9, 17]. Because of low complexity, peak signal to noise ratio (PSNR) is widely used in the performance of the communication protocols [17, 18]. However, some studies have presented that PSNR poorly correlates with subjective image quality [19]. Hence, in order to overcome the shortcomings of the PSNR, the other well known metrics are used [5, 9, 20]. At this point, the performance results of the studies should be comparable with the ones which are obtained with alternative metrics.

In wireless transmission, nodes and links have temporally correlated reception which causes consecutive packet loss. These losses are more harmful than the random packet losses in the image. In this context, the effect of the varying packet size should also be investigated in image evaluation process. In the image, some portions of the image may hold more information about the environment than the other parts. This area is called as ROI which should be transmitted more reliably. In order to investigate the performance of the communication accurately, the used image quality assessment (IQA) metric should reflect the effect of the ROI corruption on its evaluation result [21]. Although many image processing studies are focused in ROI concept, few studies are performed to consider the effects on ROI in image communications [22].

Considering the above, common image quality evaluation metrics are analyzed to present their impact on image evaluation process in wireless multimedia applications. In this paper, transmitted image is distorted in two ways: (1) Independent errors: The channel errors are randomly distributed over the image (2) Bursty errors obtained from real testbed loss patterns: They are used to present the link and node correlation effect on the received image. Many of the well known image quality methods are used on the distorted image in order to evaluate received image quality. Then, the behavior of the IQA metrics are shown in terms of ROI awareness, varying packet size and link and node correlation effect. In the light of the above mentioned investigations, the network image transmission performance is also given in terms of quality, delay and energy for varying number of flows.

The rest of the paper is organized as follows: Section 2 introduces related work. Section 3 describes the models and assumptions. Section 4 presents the evaluation of the image quality metrics in terms of correlation and packet size. Section 5 presents the performance of the MMSPEED routing protocol in terms of image quality, delay and energy. Finally, conclusion and future work is given in Sect. 6.

2 Related work

When transporting an image over a wireless channel, transmitted data are exposed to losses or errors due to channel impairments. Wireless link quality fluctuates dramatically over time, dependent on the antenna’s radiation characteristic, the distance between nodes, diffraction, scattering and other inconveniences. This dynamic nature of the wireless communication causes packet loss during communication. In wireless transmission, links have temporally correlated reception. Hence, packet losses are followed by a group of consecutive successful transmissions or vice versa, as given in Fig. 1. However, in the simulations, the packet losses over different links are independent random variables as shown in Fig. 2. This definitely eases up the analytical modeling of the energy efficiency and robustness, on the other hand, may be a strong assumption for realistic sensor networks. Analyzing the effect of packet loss pattern on the reconstructed multimedia quality is critical for designing and accurately predicting the performance of multimedia communication.

Fig. 1
figure 1

Transmitted images at different packet error rates in testbed

Full size image
Fig. 2
figure 2

Transmitted images at different packet error rates in simulation

Full size image

The effect of the packet length on energy and throughput is researched in many studies [23–25]. A few studies considered the impact of packet loss pattern and burst length in multimedia communication [26, 27]. In [27], the authors examine the distortion for independent packet loss and model the mean-squared error (MSE) expected distortion for multiple losses as being proportional to the number of losses that occur, average packet error rate. This model is acceptable when the losses can be considered to have independent effects where packet inter-arrival times are large enough [28]. In [26], to estimate the expected distortion, proposed model explicitly considers the effect of different loss patterns, including burst losses and independent (non-consecutive) losses. The study shows that burst loss pattern generally produces a larger distortion than an equal number of independent losses in video communication. Apostolopoulos [29] also supports that the length of a burst loss has an important effect on the resulting distortion, where longer burst may cause significantly more damage in the communication.

Image transmission algorithms should fulfill the perceptual quality requirements of wireless image transmission. In order to judge the quality of the transmitted image in a more realistic manner, proper metrics are required to measure the perceptual quality. In transmission of the image, communication parameters such as packet error rate and signal to noise ratio may not present the transmitted image quality. Hence, the communication protocols may require using an image quality metric to provide end to end perceptual quality by human visual system (HVS). By doing so, in addition to providing better correlation with human perceived quality, image quality monitoring and resource management for wireless imaging services is also required. There are three types of image quality assessment methods in image processing algorithms: No-Reference (NR) [30], Reduced-Reference (RR)[31], and Full-Reference (FR) metrics [32]. This classification is based on availability of the original image which is used to compare to the distorted image. As no-reference metrics try to model the judgment of image quality without the original image, reduced reference metrics rely on the low bandwidth information extracted on the original image. Most common approaches for quality measurement are FR metrics which are frequently studied in the literature [33, 34].

A well known FR metrics is PSNR which is used as an image quality metric for evaluating image processing algorithms. Many study use PSNR due to its simplicity and widespread usage [5, 18, 35]. These studies also try to avoid loss of generality by using PSNR. However, it is known that PSNR does not always rank quality of an image in the same way that a human being would [36]. Hence, in order to overcome shortcomings of the above mentioned measures, we use additional quality metrics such as weighted signal noise ratio (WSNR) [32], visual information fidelity (VIF) [37], structural similarity index (SSIM) [38] to compare their image quality prediction performance in WMSN. Here, depending on the communication layout, image loss pattern and burst length, our study tries to enable WMSN researchers to compare their performance results in terms of one of the IQA techniques (PSNR, WSNR, SSIM, VIF) to the other one.

Depending on the application, some parts of an image may hold more information. This area is called as region of interest (ROI). ROI also influences viewer’s subjective feeling since artifacts on a ROI are much more noticeable than those appearing on an inconspicuous area [39]. However, little study has been taken on identifying how and to what extent ROI will influence image quality measurements in WMSN. Although the reference image usually provides much better prediction performance than NR and RR methods, quality awareness before the whole image transmitted is extremely important in order to satisfy the application layer perceptual quality. Hence, a simple image evaluation method named Block Based Weighted Peak Signal to Noise Ratio (WBBSNR) is also proposed and evaluated in this paper. Because wireless sensor nodes are inherently resource constrained in terms of energy and computation power, WBBSNR is simple and scalable. WBBSNR is extensively investigated to present whether it can reflect the real time image quality before the communication between the source and the sink ends.

WBBSNR algorithm utilizes a prioritized block based approach to estimate the image quality before the whole image reached to the sink. In this algorithm, the blocks included in ROI area are assigned higher priority. Hence, each lost block in ROI area causes a sharper decrease of the received image quality than the other blocks which are not located in ROI. In here, WBBSNR uses block averages and packet priority in order to predict the image quality.

Considering the above mentioned constraints, this paper examines the question of whether the image loss pattern, and in particular the burst length, is important for image transmission. The effects of independent losses and burst losses may be very different in terms of image quality assessment metrics. We study the behavior of the IQA metrics applied on degraded images for the following research questions: (1) In order to explore an appropriate image quality metrics for WMSN applications, the effects of the packet loss patterns on the obtained image quality are investigated (2) We investigate the impact of both bursty and independent losses on IQA metrics in order to expose whether their behaviors change or not. (3) The applications may require knowing the quality of the ROI parts in the distorted image rather than other parts. Which image quality metrics are more sensitive to the ROI parts in image quality evaluation?

WMSN is used for practical applications in real world environments. In many cases, multimedia data must be delivered subject to time and perceptual quality constraints so that appropriate observations can be made or actions can be taken. Very few results exist to date with regard to meeting real-time requirements in WMSN [13, 14, 40]. MMSPEED protocol [14] is a hop by hop routing algorithm which does not require routing tables. All QoS algorithms in MMSPEED work locally without global network state information and end-to-end path setup. Hence it is scalable and adaptive to network dynamics. Although MMSPEED is promising for WMSN applications, it has not provided the integration of their performance results to quality evaluation techniques.

Considering the mentioned problem, we perform a group of simulation tests to present the performance of the image transmission for varying number of flows in terms of delay, energy and image quality. We use MMSPEED protocol in the performance tests. It is modified to support packet based reliability and speed options to transmit an image from the source to destination so that prioritized packets and non-prioritized packets in a flow can be differentiated. Then, the quality of the transmitted image is evaluated in terms of PSNR, WSNR, SSIM, VIF and WBBSNR. We also give the energy and delay performance of the transmitted images to present the performance of the network for varying packet size and number of flows.

3 System description

3.1 General WMSN scenario

Sensor network based homeland security systems constitute an active research field for WMSN due to their importance, challenges and inherent complexity. We consider a WMSN based border surveillance system entailing two types of sensors; Type 1 \(\hbox{C}_i, \hbox{i}=1, 2,\ldots, W\) sensors are camera nodes and Type 2 \(\hbox{S}_{j}, j=1, 2,\ldots,k\) sensors are relay sensors, depicted in the Fig. 3.

Fig. 3
figure 3

General WMSN scenario

Full size image

In the design of the camera nodes, two components are required: camera board and sensor mote. We assume that C i imaging sensors are equipped with CMOS cameras which can be interfaced with commercially available sensor platforms such as MicaZ , TelosB or Tmote Sky. Two such commercially available multimedia platforms for sensor motes are ”Cyclops” and ”CMUcam3” [1, 4].

Cyclops is an electronic interface between a CMOS camera module and a wireless mote such as Mica2 or MicaZ. It comprises of an Agilent ADCM-1700 CMOS camera module from Agilent Technology, a Xilinx FPGA and an ATMega128 microcontroller. The Cyclops communicates with MicaZ via I2C bus and uses a 2.4GHz CC2420 radio chip as the wireless component. The other platform, CMUCam3 camera card supports a set of built-in image processing algorithms, including JPEG compression, frame differencing, color tracking, histogramming, and edge detection.

Considering these commercially available camera sensor platforms with image processing capabilities, ROI localization can practically be employed by using several lightweight algorithms. For example, two cost-effective algorithms based on edge and entropy measures are used in order to identify ROI parts of the images in [41]. These algorithms are applicable to the image-blocks without requiring the whole image .

In this scenerio, if one or more intruders cross the border, one or more of C i (s) will capture the image of the intruder(s). Then they will send these images to the sink via sensors S j , which are used for forwarding data to the sink. For example, Fig. 3 portrays a scenario where C i-1, C i , C i+1 detect the intruders, and one of them (or more) will send the image or images to the sink by identifying the ROI parts of the captured image. The used test image and its region of interest area are given in the Fig. 4.

Fig. 4
figure 4

Original image and its region of interest area (ROI)

Full size image

3.1.1 Assumptions

Assumptions used for the general scenario are listed as follows:

  1. 1.

    C i ’s capability is higher than that of the S j ’s in terms of energy, processing power, and storage capacity.

  2. 2.

    The energy issue is not the primary problem for C i and the sink nodes.

  3. 3.

    S j do not die while receiving or transmitting the packets heading to it during a communication interval.

  4. 4.

    Sink’s capability is higher than all other kind of the sensors.

  5. 5.

    The sink and C i nodes don’t die during the communication. This assumption is based on the premises that these nodes are relatively more complex and powerful sensor nodes.

  6. 6.

    Each C i node transmits its image without exposing to any combining operation with the other images captured by the other C i nodes.

  7. 7.

    ROI parts of the image is identified by a low complexity algorithm, presented in [42].

3.2 Channel and radio model

The channel model links the average packet error rate with physical channel impairments. In this study, we adopted a channel and radio model from [43] applicable to real WMSN settings. We assume a shadowing channel [44] to obtain the packet error rates. Packet error rate (p) at a distance (d) for the encoding and modulation assumed in this analysis is given as follows: The packet error rate is computed as a function of the distance d between the nodes. The signal to noise ratio (SNR) is defined as

$$ \gamma(d)=P_{out}- PL(d_0) -10c\log_{10}\left({\frac{d}{d_0}}\right)+X_\sigma -P_c $$
(1)

where γ(d) is the SNR for a given (d), d is the Transmitter–receiver distance, P out is the output power of the transmitter. P c , c, X σ, d 0, PL(d 0) are constants, which are defined as the noise floor, the path loss exponent, a zero mean Gaussian (in dB) with standard deviation σ (multipath effect), a reference distance and the power decay for this distance, respectively. Assuming NRZ encoding and NCFSK modulation as utilized in various practical WMSN systems [45], packet error rate (p) at a distance (d) is given as follows:

$$ p = 1-{\left(1-{\frac{1}{2}} \exp^{-{\frac{\gamma(d)}{2}}{\frac{1}{0.64}}}\right)}^{8f} $$
(2)

where f is the frame size in bytes.

3.3 Testbed

In order to compare the image transmission quality results to the results obtained from the simulations, an indoor, 10-hops testbed by using 20 Tmote Sky sensor nodes [46] is setup. This device is equipped with CC2420 radio transceiver, MSP430 microcontroller, and ST M25P80 flash memory. TinyOS v2.1 with nesC v1.3 [47] is utilized to realize a simple still-image transmission over a chain topology. Node deployment and image transmission setup is described extensively in [16]. In this paper, we use the packet loss patterns obtained from the testbed.

3.4 Image quality metrics

In this study, five image quality metrics are used as performance criteria to measure packet loss effect on transmitted image quality: PSNR, WSNR [32], VIF [37], SSIM [38], and WBBSNR.

PSNR is calculated with the mean squared error (MSE), computed by averaging the squared intensity differences of distorted and reference image pixels, along with the related quantity of PSNR. As PSNR is simple to calculate, has clear physical meanings, and is mathematically convenient in the context of optimization, it is a commonly used metric. However, PSNR may not very well match with perceived visual quality [19]. The simplest implementation of this concept is the MSE, which objectively quantifies the strength of the error signal. But two distorted images with the same MSE may have very different types of errors, some of which are much more visible than others [48].

WSNR is the second metric which is a linear spatially invariant approximation of the HVS. It is an efficient objective quality measure in image processing applications [32].

SSIM approach is based on extracting structural information from images. Hence, a measure of the structural information change can be used to quantify perceived distortions [37]. SSIM is derived by capturing the information loss of image structures, while VIF employs the mutual information between the original and test image to evaluate the image quality. In [49], it has been demonstrated that SSIM and VIF have similar performances.

3.4.1 Peak signal to noise ratio (PSNR)

In this metric, we measure image quality distortion comparing the input image of the source’s encoder against the impaired image of the destination’s (sink) decoder. The image quality is measured as the MSE value which is defined as

$$ MSE = {\frac{1}{N_1 \times N_2}}\sum^{N_1}_i\sum^{N_2 }_j\left[I(i,j) - \widehat{I}(i,j)\right]^2 $$
(3)

where, N 1 × N 2 is the number of pixels in an image, I(i, j) and \(\widehat{I}(i,j)\) are the pixel value of the reconstructed image from the input of the source code’s image encoder, and that at the destination’s decoder, respectively. We use the following PSNR metric:

$$ PSNR(dB) = 20\log_{10}{\frac{2^s-1}{RMSE}} $$
(4)

where s is the largest possible value of the signal (s = 8, i.e. 2s − 1 = 255 for grayscale images), and RMSE is the root mean square error between the two images given above, respectively.

3.4.2 Weighted signal to noise ratio (WSNR)

WSNR uses a frequency domain transform function naming contrast sensitivity function (CSF). CSF is utilized to filter spatially all inappreciable frequencies by the human visual context. This quality measure can take into account the effects of image dimensions, viewing distance, printing resolution, and ambient illumination. First step for calculating WSNR is to find an error image by computing the difference between original image and distorted image. Then, the error image is weighted by a linear spatially invariant approximation to frequency response of the HVS given by CSF. Finally, WSNR is computed [32]. This procedure can be given as

$$\hbox{ WSNR(dB) }= 10\log_{10}\left[{\frac{\displaystyle\sum_{i,j} \left|I_{f}(i,j)CSF(i,j)\right|^2}{\displaystyle\sum_{i,j} \left|\Upomega(i,j)CSF(i,j)\right|^2}}\right] $$
(5)

where

$$ \Upomega(i,j) = I_{f}(i,j)-\widehat{I}_{f}(i,j) $$
(6)

and \(I_{f}(i,j), \widehat{I}_{f}(i,j)\) and CSF(i, j) represent the Discrete Fourier Transform (DFT) of the input image, reconstructed image and CSF, respectively.

3.4.3 Visual information fidelity (VIF)

VIF [37] views image quality assessment problem as an information degradation of visual quality due to a distortion process. This approach attempts to relate signal fidelity to the amount of information that is shared between reference and distorted image signals. In VIF model, the reference image is modeled by a wavelet domain Gaussian scale mixture (GSM), which has been shown to effectively represent the non-Gaussian marginal distributions of the wavelet coefficients of natural images, while also capturing the dependencies between the magnitudes of neighboring wavelet coefficient.

3.4.4 Structural similarity index (SSIM)

The SSIM index [38] is evolved to the assumption that HVS is based on extracting structural information from an image. Structural information is independent of the illumination of an image. SSIM index compares two image patches which are taken from the same location to predict structural degradations. The structure degradation is computed after normalizing the image patches for mean luminance and contrast. Comparison operation on the image patches is based on finding similarities between their local intensities, contrasts and structures. Suppose that x and y are local images. The final SSIM index is expressed by

$$ SSIM(x,y) = {\frac{(2\mu_x\mu_y+C_1)(2\sigma_{xy}+C_2)} {(\mu_x^2+\mu_y^2+C_1)(\sigma_x^2+\sigma_y^2+C_2)}} $$
(7)

where μ x and μ y are the mean intensity of image x and image y respectively, σ x and σ y are contrast of image signals respectively, and σ xy denotes the covariance between x and y. The constants C 1 and C 2 are utilized to refrain instabilities in the structural similarity comparison that may occur for certain mean intensity and contrast combinations.

3.4.5 Weighted block-based peak signal to noise ratio (WBBSNR)

In this paper, an image quality assessment metric, WBBSNR is proposed for measuring of visual quality impairments influenced during wireless sensor network communication. It deals with transmitted image quality by considering packet loss nature of the WMSN. Generally, all image quality metrics in literature focus on distortions due to compression, dithering, and printing effect on the quality of the image. However, transmission of the images over WMSN channels is packet based. Hence, the impact of packet loss on image quality assessment is primary issue for WMSN. Also, depending on the application in WMSN, the interest of area in the image may need more reliability due to contained information in. Therefore, the transmission distortion within these attended areas should be considered more severe than elsewhere [50]. In this context, the transmission of the background area may also be required in order to analyze the full environment in detail. Hence, image quality assessment method should consider this issue by assigning different weights to the ROI and Non ROI (background) parts of the image in the quality evaluation phase.

In WBBSNR, images are partitioned into the s × s blocks. The averages of the blocks in the image are found. Then blocked mean squared errors are calculated separately for both ROI and Non ROI area. It is given by

$$ BMSE_{l} = {\frac{1}{M \times M}}\sum^{M}_i\sum^{M}_j\left[N_l(s_i,s_j) - \widehat{N_l}(s_i,s_j)\right]^2 $$
(8)

where l \(\epsilon \{R,NR\}.\) In here, l = R depicts the ROI area, l = NR background area, BMSE l is block mean square error, M 2 is the number of blocks in an image, N(s i , s j ) and \(\widehat{N}(s_i,s_j)\) are the average of the block’s of the reconstructed image from the input of the source code’s image encoder, and that at the destination’s decoder, respectively.

In the calculation of WBBSNR, ROI and background of the image (ROI and NonROI) is calculated separately. It is given as:

$$ WBBSNR_l(dB) = 20 \log_{10}{\frac{2^c-1}{\sqrt{BMSE_{l}+\theta_l}}} $$
(9)

where c is the largest possible value of the signal and θ l is a regulatory parameter, very small number, in order to prevent a divide-by-zero condition.

Then overall WBBSNR is given by

$$ WBBSNR=\lambda _{R}WBBSNR_{R}+ \lambda_{NR}WBBSNR_{NR} $$
(10)

where λ R is the weighting coefficient for the ROI part of the image and λ NR is the weighting coefficient for the background part of the image. In the selection of these weighting coefficients, the following equation must be satisfied:

ROI distorted test images, which are given in the Fig. 5, are obtained by applying with different packet loss pattern. Then, WBBSNR evaluations corresponding to these distorted images are performed, where all possible combinations of the λ R coefficient are integrated with WBBSNR. Consequently, obtained WBBSNR qualities and the impaired images are visually diagnosed for finding the best (λ R ) for the considered scenario.

$$ \lambda _{R}+ \lambda_{NR}=1 $$
(11)

Here, in order to pay more attention to visual quality of ROI parts of the image, (λ R ) should be greater than (λ NR ). As shown in the Figs. 5 and 6, higher (λ R ) leads to put more emphasis at ROI parts of the image in the measured image quality and vice versa. According to these empirical tests, it is suitable for the considered scenario that λ R and λ NR are selected as 0.7 and 0.3, respectively.

Fig. 5
figure 5

ROI distorted test images for five different error rates. a Error rate (%) = 1, b Error rate(%) = 3, c Error rate(%) = 5, d Error rate(%) = 7, e Error rate(%) = 9

Full size image
Fig. 6
figure 6

WBBSNR versus error rate for different lambda R coefficients

Full size image

4 Performance analysis of IQA techniques

In this section, three types of performance tests are given to illustrate the effect of wireless sensor network nature on image quality: packet size, ROI susceptibility and link correlation. Once the image is obtained to be transmitted to the sink, packet loss patterns obtained through the channel model are applied, in order to eliminate the lost pixels from the original image.

To examine the performance of transmission algorithms, we have conducted comprehensive sets of simulation experiments in MATLAB. The considered WMSN setup follows the scenario described in Sect. 3. In the simulations, we have used five gray-scale test images. The selection criteria of the test images are their suitability to the border surveillance applications.

In this part of the simulations, only one source generates image data and this image is transmitted to the sink. The used test image and its region of interest area are given in the Fig. 4. The figures containing IQA results versus image error rate is based on this test image. The packet loss patterns obtained through the simulations and testbed are applied to the test image in order to constitute the distorted ones. Then, image quality assessment metrics (PSNR, WSNR, SSIM, VIF and WBBSNR) are applied on them to evaluate the effect of the correlation and packet size. Three image distortion methods are used, given in Fig. 7: (1) Whole image is distorted by means of the packet loss patterns (Image) (2)The image parts excluding region of interest are distorted (Background) (3) Only region of interest parts of the image is distorted (ROI).

Fig. 7
figure 7

Transmitted images for three different distortion models

Full size image

Many studies are done related to the packet loss over wireless networks which cope with determining the optimal packet size based on different metrics, such as throughput, goodput, transmission range, and energy efficiency [25, 51–53]. In this study, we investigate packet size impact on well known IQA metrics in order to explore their behaviors for varying error rates. Hence, our study leads to make a comparison among image communication studies which may use different packet size, IQA metrics and environment (simulation, testbed).

We repeated each simulation 50 times with different packet loss patterns where different packet sizes (ps) are used (16, 64, 256). The obtained image quality results are processed to calculate the average quality of the transmitted image. Table 1 shows our performance metrics used in this part of the simulations.

Table 1 Performance metrics
Full size table

In the simulations, we used the channel and radio model given in Sect. 3.2. Packet error rate is derived from the physical layer characteristics such as output power and distance. To acquire the constant packet error rates in simulations, we have tested several experiments to find proper configuration by varying distance between nodes. The packet loss probabilities are assumed to be independent among links, and identically distributed (i.i.d.) random variables. There is no spatial correlation of losses [28]. Furthermore, in order to explore the correlation effect on the IQA techniques, the packet loss patterns which are coming from a real testbed are used. The explanation about the testbed is also given in the Sect. 3.3.

The Pearson product-moment correlation coefficient denoted by r is also used to present the correlation between the image error rate and each IQA metric. The stronger the association of the two variables the closer the Pearson correlation coefficient, r, will be to either +1 or −1 depending on whether the relationship is positive or negative, respectively. A value of 0 indicates that there is no association between the two variables. Here, all IQA results for the five test images are considered together in order to obtain Pearson coefficients.

4.1 Packet size and ROI awareness

Many studies were performed on energy optimization for changing packet size. However, the effect of the packet length on transmitted image quality has not been studied as extensively. In this study, IQA metrics’ behaviors against varying packet sizes are analyzed for transmitting images in WMSN in order to determine the application layer image quality requirements. After capturing the image in the camera node, image is converted to data packets in order to transmit to the sink. Depending on the underlying network layout, several packet sizes are used. Due to wireless channel fluctuations, packet losses frequently occur during transmission. Packet size may also affect the evaluated image quality at the end node depending on the image quality assessment method used. We analyze the packet size effect on the quality of the image in terms of PSNR, WSNR, SSIM, VIF and WBBSNR. In this part, we use the channel error model explained in the Sect. 3.2 in the simulation.

Figure 8 shows that image quality versus error rate in terms of PSNR, WSNR, SSIM,VIF, WBBSNR for different packet sizes (ps = 16, 64, 256). The image errors occur on the image uniformly, not depending on the importance of the part of image data being transmitted. Figure 8(a) presents the PSNR performance for varying image error rates. For all packet sizes, PSNR shows the same behaviors. It starts from nearly 26 dB and goes to 16 dB. We also know that the acceptable quality for PSNR is approximately 20–17 dB in wireless networks. 10 –15 % packet losses cause the received image quality to be unacceptable. It is shown that packet size does not affect the received image quality in terms of PSNR.

Fig. 8
figure 8

IQA performances versus image error rate where errors are distributed randomly on the image. a PSNR versus image error rate, b WSNR versus image error rate, c SSIM versus image error rate, d VIF versus image error rate

Full size image

Figure 8(b) shows WSNR performances of the images for varying error rates for different packet sizes. WSNR behaves differently from the other IQA metrics. As packet sizes are increasing, the image quality decreases where test images are impaired with the same error rate. For example, when image error rate is 10 %, WSNR presents 13.06, 11.48 and 10.82 dB image quality results for the packet sizes 16, 64, 256, respectively. As a result of this phenomenon, WSNR operational thresholds are determined depending on the packet size. We found that WSNR operational threshold value can be chosen as 17–14 dB which is the lowest image quality bound of the transmitted image.

Due to the use of the structural information of the image such as mean, variance etc, SSIM is a very common metric to capture better perceptual quality as compared to PSNR and WSNR. Figure 8(c) presents SSIM performance of the images for varying error rates. As opposite to WSNR, when the packet size is increasing, SSIM performance increases for the corrupted images at the same error rate. That means there is positive correlation between packet size and the image quality in terms of SSIM. For example, SSIM drops from 0.95 to 0.64 in response to an increase of error rate from 1 to 10 % when packet size is 16. Although the image is corrupted at the same error rate, the performance results of the SSIM for ps = 256 gives from 3.2 to 31.2 % increment over the results for ps = 16. Hence, SSIM gets different threshold values depending on the packet size. The operational thresholds of the SSIM are approximately 0.8, 0.9 and 0.95 for packet sizes with 16, 64, 256 respectively. Hence, network aware services, in order to provide acceptable quality during transmission, should be adaptive to such changes.

Figure 8(d) shows the VIF image quality versus error rate for ps = 16, 64, 256. As increasing error rate, VIF moves from 0.8 to 0.2 which will result in 75 % performance reduction for ps = 16. However, when the packet size is increased, VIF behaves similar to SSIM. There is a positive correlation between packet size and the image quality in terms of VIF. When the error rate increases, its performance reduction will be 41.9 and 25 % for ps = 64, 256 respectively. Although the image is corrupted at the same error rate, the performance results of the VIF for ps = 256 represents 20–260 % increase over the results for ps = 16. As a result of these values, it can be said that VIF is more sensitive than SSIM for varying error rates and packet sizes. The operational thresholds of the VIF is approximately 0.6, 0.8 and 0.9 for packet sizes with 16, 64, 256 respectively.

Figure 9 shows image quality assessment methods versus image error rates where the errors are distributed on the ROI part of the image. In this figure, we aim to present the behavior of the IQA metrics in terms of region of interest awareness. Figure 9(a) shows PSNR performance for ROI corrupted image when increasing the error rate. Figure 9(a) shows that PSNR performance decreases approximately 2 dB as compared to the one obtained in the Fig. 8(a) for all packet sizes. Due to nature of the PSNR, It may not be accurate to make a general statement that packet losses in ROI area adversely affect PSNR performance. As pixel values in ROI area are closer to 255 for gray-scale image, the PSNR result will be worse or vice versa. Figure 9(b) shows WSNR performance of the ROI corrupted image for different error rates. When increasing error rate, WSNR performance increases from 19.83 to 9 dB for ps = 64. Both PSNR and WSNR are calculated based on the signal to noise ratio. Hence, pixel values in the image play an extremely important role. When we analyze the histogram of the ROI parts in the image, there is a huge pixel group whose value is bigger than 150. On the contrary, the pixel value in the background parts of the image is clustered under 150 as shown in the Fig. 10(b). As a result of this tests, we have a conclusion that SNR based methods are tightly coupled with the image properties [54].

Fig. 9
figure 9

IQA performances versus image error rate where the errors are distributed randomly on the parts of image which are located in ROI area. a PSNR versus image error rate, b WSNR versus image error rate, c SSIM versus image error rate, d VIF versus image error rate

Full size image
Fig. 10
figure 10

Histogram of the test image. a Histogram of the ROI parts in the image. b Histogram of the background parts in the image

Full size image

Figure 9(c) presents SSIM image quality performance for ROI corrupted image under varying error rates and different packet sizes. SSIM quality variation decreases significantly in this Figure. For example, when the packet size is 16, SSIM performance starts from 0.96 to 0.85. Furthermore, SSIM gives the same performance which starts with 0.97 and goes to 0.85 for both packet sizes. When the error rate is approximately 10 %, SSIM gives greater improvement 32.8, 9 and 1.2 % over the performance in the Fig. 8(c). When the packet size is increasing, the gap between ROI and equally corrupted image performances closes, in terms of SSIM. Figure 9(d) also shows that SSIM and VIF behave similarly. All of the image test results present that as increasing error rate, VIF performance variation decreases for all of the packet sizes. The operational thresholds for SSIM and VIF should also be increased to 0.95 and 0.85, respectively.

Figure 10 presents a histogram of the pixel intensity values of the used test image which is shown in Fig. 4. Thus, it presents the distribution of pixels in the used images, which has a value from 0 (which is black) to 255 (which is white). The left side of the histogram shows the black end of the scale and the right side represents white.

Table 2 presents the correlation between IQA performances and error rates for the test images when the losses are in the whole image, background image and ROI parts of the image. In Table 2, SSIM and later VIF are the most correlated metrics in terms of the error rate. Table 2 presents VIF is the most affected metric by packet size and ROI loss.

Table 2 Pearson correlations between error rate and image quality for varying packet size (95 % confidence interval for r, significance level \(\prec\) 0.00001, sample size = 500)
Full size table

It is expected that ROI losses decrease the VIF performance. On the contrary, we have observed an increase on the quality in terms of VIF metric. Table 3 also enables a comparison between simulation quality results and testbed quality results in terms of Pearson correlations, where packet size is 16. Table 3 also presents that VIF and WBBSNR are the most affected metrics by the packet loss pattern which comes from testbed or simulation. It is also show that SSIM metric has the best correlation with the error rate in the testbed.

Table 3 A comparison of both testbed and simulation quality results in terms of Pearson correlations between error rate and image quality ((95 % confidence interval for r, significance level \(\prec\) 0.00001, sample size = 500)
Full size table

All IQA data points from the five images are considered together in order to obtain Pearson coefficients for Tables 2 and 3. As shown in the Tables 2 and 3, obtained confidence intervals for this group of the images are highly acceptable. However, if more sources of different content were jointly assessed, the Pearson correlation would be much lower than this.

4.2 Performance analysis of the WBBSNR metric

In this paper, a simple metric named WBBSNR is proposed to obtain the image quality monitoring before the test image is reached to the destination.

In this paper, the ROI percentage of the selected images is nearly 10 %, and background parts is 90 %, remaining part of the image. Figure 11 presents the WBBSNR performance versus equally, ROI area and Background area corrupted images separately. In order to show WBBSNR behavior for varying error rate and to compare the WBBSNR behaviors between ROI and equally corrupted image regions accurately, Fig. 11(a) and (c) should have the same order of magnitude. However, Fig. 11(b) implies the distortions on background parts of the images. In order to analyze the behavior of the WBBSNR in depth, we select its order of magnitude from 10 to 90 %. Hence, if all of the background parts are distorted, the healthy blocks are composed of the ROI parts, as shown in the Fig. 11(b).

Fig. 11
figure 11

WBBSNR performances versus image error rate where errors are distributed randomly on the image. a WBBSNR versus image error rate for equally corrupted images, b WBBSNR versus image error rate for background corrupted images, c WBBSNR versus image error rate for ROI corrupted images

Full size image

Figure 11(a) shows equally corrupted image quality evaluation for all packet sizes when the error rate increases. As mentioned in the Sect. 3.4.5, this metric has also coefficients which provide to emphasize some blocks of the images over other ones. Note that they may also change depending on the application. WBBSNR also behaves like SSIM and VIF for increasing packet size. Its performance increases with the packet size as it is positive correlated due to its block based nature. As increasing of the error rate, its performance decreases by 91.8 and 84.1 % for ps = 64 and 256, respectively. Although the image is corrupted at the same error rate, the performance results of the WBBSNR for ps = 256 gives from 58.8 to 206.1 % increment over the results for ps = 64.

Figure 11(b) shows the WBBSNR performance of the image with background parts corrupted versus image error rate for different packet sizes. This figure presents that WBBSNR is robust for losses in the ROI areas. Its performance decreases from 48 to 28 dB when the error rate increases approximately from 0.9 to 89 % for ps = 16. As increasing the packet size, the performance of the WBBSNR increases as expected. As the error rate increases, the improvement of the WBBSNR for ps = 64 is from 12.5 to 21.4 % over the performance with ps = 16. Due to prioritization coefficient used, it is expected that WBBSNR performance in this scheme is better than Fig. 11(a) and (c).

Figure 11(c) presents the ROI corrupted image performance versus error rate for the different packet sizes. As compared to Fig. 11(b), performance reduction is shown clearly. However, it gives better performance than the WBBSNR in Fig. 11(a). Because the number of none prioritized blocks are clearly more dominant for equally corrupted images. Hence, the adjustment of the thresholds should also be chosen more sophistically.

4.3 Link and node correlation effect on IQA methods

In this part, we aim to present the link and node correlation effect on the image quality assessment methods. The node and link correlation causes burst losses during transmission. Figures 12 and 13 present that the effect of the channel errors and burst losses on the image quality performances in terms of PSNR, WSNR, VIF, SSIM and WBBSNR for ps = 16. Figure 12 presents that whole image is corrupted by means of the testbed loss patterns and the channel errors used. To compare the results coming from the channel and real testbed image quality results, we give the all results for ps = 16 in Fig. 12.

Fig. 12
figure 12

IQA performances versus image error rate where simulation errors are randomly distributed on the parts of image. a Testbed and simulation comparisions in terms of PSNR, WSNR and WBBSNR for decreasing error rate. b Testbed and simulation comparisions in terms of SSIM and VIF for decreasing error rate

Full size image

As shown in the Fig. 12(a), PSNR testbed results for PSNR tb are nearly the same as PSNR where the corrupted image is provided by the channel errors. PSNR tb starts from 26–16 dB for increasing error rates. As similar to PSNR tb , WSNR tb also gives a close quality performances to the WSNR channel model results. As shown in this figure, as WBBSNR tb channel performance presents performance fluctuations for increasing error rate, WBBSNR gives more stable performance reduction. WBBSNR tb starts from 34 to 1 dB for increasing error rate.

Figure 12(b) presents VIF and SSIM performance for corrupted images in the channel and testbed. SSIM testbed quality results (SSIM tb ) moves from 0.96 to 0.76 and gains from 1.1 to 26.7 % improvement over SSIM channel model results. As similar to the SSIM tb performance, testbed VIF performance VIF tb has also performance improvement from 5.0 to 48.3 % over VIF channel results.

To give the IQA performance of the impaired ROI areas in case of the burst and channel loss, we give the Fig. 13 for ps = 16 when the error rate is increased. Figure 13(a) shows that PSNR tb performance starts from 25.33 dB to 14.25 for increasing error rate. PSNR tb and PSNR channel performance is nearly the same in this figure. WSNR tb performance has a behavior similar to PSNR tb as expected. WSNR and WSNR tb show very similar performance as the error rate increases. As compared to the whole image corruption of the WBBSNR tb in the Fig. 12(a), ROI corrupted WBBSNR tb performance presents worse image quality from 9 dB when the error rate is 0.1.

Fig. 13
figure 13

IQA performances versus image error rate where simulation errors are distributed randomly on the parts of image which are included in ROI. a Testbed and simulation comparisions in terms of PSNR, WSNR and WBBSNR for decreasing error rate. b Testbed and simulation comparisions in terms of SSIM and VIF for decreasing error rate

Full size image

Figure 13(b) gives VIF and SSIM performance for increasing error rate. In this figure, SSIM tb shows the SSIM performance in case of the burst loss provided by the testbed. It is interesting that SSIM tb presents similar performance SSIM channel performance results. SSIM tb performance starts approximately from 0.97 to 0.86. VIF tb gives 8.9 % improvement over VIF channel performance when the error rate is 1.06 %. When increasing error rate, the performance of VIF tb and VIF channel performance gets closer and eventually show the same image quality value (0.85).

5 Performance analysis of IQA techniques with a underlying routing protocol

In this section, to analyze the performance of image quality metrics and packet size with an underlying routing protocol (MMSPEED) [14], we performed a group of simulations using J-SIM network simulator [55]. Table 4 shows the MMSPEED simulation parameters used in J-Sim. In our work, a packet train traffic model is used between one or more sources and single destination, which is common in WMSN applications. The packets in a packet train are transmitted back to back by a node without releasing the channel which causes the burst losses or vice versa. In this work, 100 nodes are placed in a 100 m × 100 m region. We set the transmission range to 40 m, the bandwidth 250 kbps. In the simulations, we used a gray-scale image which is shown in Fig. 4. We repeated each simulation 15 times with different performance metrics. The simulation results are used to calculate the average quality of the transmitted image in terms of PSNR, WSNR, SSIM, VIF and WBBSNR.

Table 4 System and simulation parameters
Full size table

In this part of the analysis, it is assumed that there are one or more sources which may generate image data. In the communication, when the number of sources are greater than one, the received image quality is evaluated the results coming from the one of the sources. End-to-End packet delay, packet delivery performance, normalized energy consumption are the metrics used to analyze the performance of image transmission.

The service differentiation of the MMSPEED is actually based on inter-prioritization of different data types i.e. scalar data, audio, video etc. However, image transmission requires the intra-image prioritization in order to assign different priorities to the packets. Hence, MMSPEED routing protocol is also modified to perform intra image prioritization based on the ROI areas of the image. In the performance results, transmitted image blocks are prioritized depending on whether the packet is in ROI area or not. The packets in the ROI area are prioritized as high priority packets or vice versa. Hence, their required reliability and deadline constraints are tried to be satisfied by the network layer. The requirements for low and high priority packets are also given in the Table 4. We give the performance of MMSPEED routing protocol for varying number of flows in terms of PSNR, WSNR, SSIM, VIF, and WBBSNR.

End to end delay is a crucial metric for WMSN. Multimedia data may require that each packet should reach its destination by the deadline. Hence, in this paper, we analyze the delay performance of the image transmission for increasing number of flows. End-to-end delay is the average delay between sending the data packet by the source and its receipt at the destination. The delay includes the elapsed time caused by route acquisition, buffering and processing at intermediate nodes, and retransmission delays at the MAC layer for a given simulation time.

Multimedia applications generate huge data delivered over WMSN, resulting in the difficulty for both reliable communication and network lifetime especially considering the scarce energy constraint of the sensor nodes. Hence, energy is a crucial design issue in WMSN. In this paper, at the start up of the simulation, we assign a constant energy value to each node in the network. During the communication, each node updates its energy level depending on participation on the communication. Energy consumption is given for different packet sizes for increasing number of flows. Total energy consumed by all sensor nodes are retrieved for analysis at the end of the simulation. In this study, the energy parameters used in the simulations are shown in Table 4. The effect of packet sizes on the communication performance is also investigated in terms of delay and energy. Figure 14 shows the image quality performances of the MMSPEED in terms of IQA techniques for varying number of flows. Figure 14(a) presents PSNR, WSNR and WBBSNR performance for varying number of flows for ps = 16 and ps = 64. When increasing the packet size, PSNR performance increases from 10 to 3 dB as expected. Because, as packet size is increasing, the packet reception rate also increases at the destination. Actually, excluding PSNR, all other metrics are affected by packet size. It is now known that the nature of the PSNR is not affected by the size of the lost packet. Hence, we can conclude that increasing the packet size in a network may cause highly distorted images at the sink. As shown in the Figure, the acceptable quality of the image is provided when the number of flows is less than 6 in terms of PSNR. This conclusion is also valid in terms of other metrics results in this figure. However, especially the operational thresholds of the WBBSNR metric for acceptable image quality should be changed depending on the packet size in order to decide whether the image has enough quality or not. Because, all of the three metrics depends on the signal to noise ratio, their performance holds the condition that they are comparable.

Fig. 14
figure 14

MMSPEED image transmission performances versus number of flows in terms of image quality. a PSNR, WSNR and WBBSNR versus number of flows for packet size 16 and 64, b SSIM and VIF versus number of flows for packet size 16 and 64

Full size image

The most affected metrics by the packet size are VIF and SSIM. Fig. 14(b) presents the performance of these metrics for increasing number of flows with different packet sizes. When the packet size is 16, SSIM performance changes from 0.98 to 0.78 while number of flows is between 1 and 12. When the packet size is increased to 64, its performance is between 0.97 and 0.53 depending on number of flows. As a result of this, distortion difference for transmitted images with different packet sizes is greater than expected due to varying packet sizes. The results should be investigated that the transmitted image qualities in terms of SSIM and VIF are not only affected the network performance but also used packet size and the location of the loss packets affects the transmitted image quality. Furthermore, if these metrics are used in the network monitoring, operational thresholds should be adjusted depending on the packet size before the transmission of the image. Figure 15 presents the delay and energy performance versus number of flows for packet sizes (16, 64). These tests are performed to consider the effect of the collisions, routing decisions, and wireless channel affects. In these results, the different packet size is used to monitor the performance of the network during transmission. The results show that when decreasing the packet size, the performance of the transmission in terms of energy is reduced. Figure 15(a) presents that smaller packet size (ps = 16) causes an increase of the delay 752 % over the one with ps = 64, when the number of flows is 16. When the number of flows is 16, this figure also presents the performance gap is lesser between ps = 64 and ps = 256. Figure 15(b) presents as increasing the packet size, the consumed energy also increases. Particularly, the energy consumption of the ps = 16 presents 295 % increase over the one with ps = 256 when the number of flows goes to 16.

Fig. 15
figure 15

MMSPEED image transmission performances versus number of flows in terms of delay and energy. a Delay of the routing protocols versus number of flows. b Total energy dissipation of MMSPEED routing protocol

Full size image

6 Conclusion

In this work, the impact of the packet size, link and node correlation effect and loss areas on evaluated image quality of a distorted image are evaluated with respect to wireless networks errors. The performances of well known image quality metrics (PSNR, WSNR, SSIM, VIF) are given to understand their suitability to wireless communication for a group of test images. In this paper, we also give a simple image quality estimation method (WBBSNR) based on transmitted packet averages and their importance level. It can be used to evaluate image qualities on the fly without using all of the image pixel information at the intermediate nodes. From the experimental results, this metric can estimate the image quality with a high accuracy. By means of this proposed metric, communication protocols adaptively adjust their behaviors depending on the estimated image quality offered by the network. This metric is also appropriate for the communication protocols which have a prioritization operation before communication starts up.

The performance results show that packet length and link and node correlation does not have an impact on PSNR. Hence, observed image quality results obtained from the testbed and simulation are nearly the same in terms of PSNR. PSNR also does not present a different behavior in case of packet losses in the region of interest areas of the image. The determining factor in PSNR is the pixel values of the lost packet. Hence, when the pixel values in ROI region are closer to zero, the PSNR increases or vice versa. The packet size effect on WSNR is different from all the other image quality metrics. As packet size increases, the WSNR quality of the distorted image subjected to the same error rate decreases for our test images. Real testbed and simulation loss patterns also give similar image quality results in terms of WSNR.

The effect of the packet length and correlation on SSIM and VIF are similar. As packet size increases, the presented image quality in terms of SSIM and VIF also increase. However, VIF is more sensitive to the packet size than SSIM. Furthermore, testbed image quality results are better than simulation results in terms of VIF and SSIM. The performance results also present that although SSIM and VIF are more sensitive than PSNR and WSNR in terms of ROI awareness, the quality of the ROI distorted image is increased in terms of SSIM and VIF.

As further improvement, prevalent image quality metrics (PSNR, WSNR, SSIM and VIF) can be modified to increase the effect on the ROI parts on the evaluated image quality. IQA metrics can be combined with a weighting operation in order to represent overall image quality. Another suggestion would be that the quality evaluation of the ROI and background parts of an image can be performed separately. This work may also be expanded so that the behaviors of the IQA techniques can also be analyzed for a large number of images which are classified for different multimedia applications.

Furthermore, this study leads to explore smart ways for the adaptation of the communication protocols to the wireless environment for a multimedia transmission. For example, on-the-fly image quality assessment techniques can be combined with a QoS based routing protocol to obtain the required image quality by the application layer.