1 Introduction

Multimedia services like IPTV, VoD (Video on Demand) and VoIP (Voice over IP) have been merged into the Internet since its emergence. A sharp increase of IP network bandwidth makes it possible to provide IPTV service that has a real-time requirement. Especially, IPTV service environment has been under a dramatic change along with the IPTV market growth and subscribers’ demanding requirements [18]. Also, QoS management architecture has been studied for QoS provision [13, 18]. As various multimedia services are provided in the integrated network environment, QoS (Quality of Service) and QoE (Quality of Experience) concepts are introduced in the IP network to describe satisfactions about subscriber’s quality requirements. However, real-time video streaming service providers and users still suffer from the legacy of IP network, the uncertainness of packet transmissions.

In these challenging scenarios, it is critical to guarantee an appropriate QoE for the end user, according to the application to be developed. As various multimedia services are provided in the integrated network environment, QoS and QoE concepts are introduced in the IP network to describe satisfactions about end user’s quality requirements. QoE can be defined as the overall performance of a system, from the user perspective. Many factors can affect the QoE, depending on the application and users expectations. Video perceptual quality is one of the most important aspects to consider in the user QoE.

QoE is the assessment of the level of customer satisfaction to the provided service. QoE effects are caused by QoS problems such as delay, jitter, loss, and bandwidth. But we don’t know very well the cause effect relationship. Traditional measurement of quality is based on only QoS. QoE is composed of not only the network performance parameter but also the service quality parameter such as cost, reliability, availability, usability, and fidelity. Although QoE is very subjective in nature, it is very important that a strategy is devised to measure it as realistically as possible. The ability to evaluate QoE will give the provider some sense of the contribution of the network’s performance to the overall level of subscriber satisfaction. The network providers can offer and maintain optimum network environment for multimedia service such as IPTV since they can forecast whether to provide multimedia services above a certain level of service quality using the proposed model.

There are numerous network-related features affecting IPTV service quality. To manage the service quality effectively, the IPTV QoE should be monitored and kept in good condition by service providers. However, QoE-related QoS parameters have the different influence and cannot be treated in a same weight. Network performance objectives are recommended by the ISO (International Organization for Standardization) to provide the IPTV service, but it is not sure that provided network guarantees user’s quality satisfaction. Thus, multimedia service QoE assessment method reflecting network environment is needed [1].

The investigation of a correlation between QoS and QoE has being progressed in order to solve the challenges of the satisfaction evaluation. According to NP (Network Performance), QoS and QoE items, the existing researches [16, 17] deduced the major quality elements of the IPTV service. They used the QDF (Quality Deployment Function) methodology for analysis of relationship between the drawn QoE items and the QoS parameters. However, they were nothing more than correlation analysis did not propose measures that can quantify QoE.

QoE challenges are to quantify the customer satisfaction and to measure what degrades customer satisfaction. In the IP-based network environment, the existing research [22] for the image multimedia service QoE measurement has considered only the IP Packet Loss and Bandwidth among the QoS quality parameters. However, the various QoS parameters causing an effect in the QoE of the IPTV service aren’t reflected in [22]. The proposed model assesses the video quality of experience by normalizing the various QoS parameters such as packet loss, jitter, delay and bandwidth. But, they measured the quality of the video by separating packet loss and jitter. For this reason; it is not adequate to evaluate the subscriber satisfaction of the IPTV service in future. In order to solve the problem of the existing researches, we develops the QoS/QoE correlation model to numerically evaluate the IPTV QoE by using relative importance of QoS parameters. Network and service providers can predict and evaluate the end users’ video QoE through simple QoS parameter measurement because our proposed model is a simple mathematical model which is capable of measuring the user video QoE using correlation between the video quality of experience and network layer QoS parameters. Through our proposed model, service provider can quantify and measure subscriber’s QoE in provided network environment and analogize service environment which meet the optimum QoE, conversely.

2 Related works

This clause addresses related works of the video service quality measurement method, studies of correlation between QoS and QoE, and QoE assessment model using QoS parameters.

2.1 Video service quality measurement method

Existing quality assessment technologies can be classified into two categories: subjective quality assessment schemes and objective ones. Current research status of these two kinds of evaluation methods are illustrated as followed paragraphs:

Currently, the evaluation methods for the speech service are mature. For subjective evaluation methods, opinion rating (MOS) based on customer’s satisfaction has been studied to assess the perceptual QoS. It is specified in ITU-T recommendations E.800 initially [8]. On another hand, several objective quality assessed methods has been proposed in ITU-T, such as P.861 [9] PSQM (Perceptual Speech Quality Measure), P.862 [10] PESQ (Perceptual Evaluation of Speech Quality) and G.107 E-Model [7].

For the video service evaluation, subjective video quality evaluation method is the most reliable video quality measurement method. A group of viewers is selected and gathered in a room, the measurement environment is specified in the ITU-T Recommendation P.910 [11]. For the research of objective video quality method, some estimation software has been developed which can analyse the video signals and produce the quality evaluation results. One traditional objective video quality measurement, Peak Signal to Noise Ratio (PSNR), has been widely used in many applications to assess video quality.

PSNR does not take the visual masking phenomenon into consideration. In other words, every single pixel error contributes to the decrease of the PSNR, even if this error is not perceived. So, MPQM (Moving Pictures Quality Metric) was proposed for the objective the video quality measurement [22, 24]. MPQM is an objective quality metric for moving picture which incorporates human vision characteristics. MPQM represents the typical image quality assessment models based on the error sensitivity. The widely adopted assumption of these models is that the loss of perceptual quality is directly related to the visibility of the error signal.

VQM (Video Quality Metric) [23] is developed by ITS (Institute for Telecommunication Science) to provide an objective measurement for perceived video quality. It measures the perceptual effects of video impairments including blurring, jerky/unnatural motion, global noise, block distortion and color distortion, and combines them into a single metric. The testing results show VQM has a high correlation with subjective video quality assessment and has been adopted by ANSI as an objective video quality standard. And, a different approach for video quality assessment is presented by Zhou Wang [25]. Their method differs from the previously described methods, which all are error based, by using the structural distortion measurement instead of the error. The idea behind this is that the human vision system is highly specialized in extracting structural information from the viewing field and it is not specialized in extracting the errors. Thus, a measurement on structural distortion should give a better correlation to the subjective impression.

To numerically evaluate prediction of the objective metrics, they calculated Pearson’s correlation coefficient between objective marks (after applying the fitting function) and subjective ones. SSIM is the best objective metric as shown in Table 1. So, we infer the user’s perceived quality level using SSIM score.

Table 1 Correlation between objective video quality metrics and MOS (Mean Opinion Score) subjective impression [3]
Full size table

From current research status of evaluation method, we can see that the subjective method based on user survey can reflect the experience of user more directly and match well to the feeling of user. However, this kind of method has several problems, such as, it required special environment and equipment, needs a mass of people to participate the test. In conclusion, subjective video quality measurement cannot provide real-time and in-service quality monitoring for real-time video applications.

2.2 Studies of correlation between QoS and QoE

Currently, the investigation of QoS and QoE correlation is continued. Khirman and Henriksen were trying to relate the objective network service conditions with the human perception of the quality of the service. Their subject has been widely investigated for voice delivery and it is widely acknowledged that the relationship between voice transmission conditions and the human perception of quality is far from linear [14]. They discuss in detail how the human satisfaction of HTTP service is affected by the two main network QoS parameters, namely network delivery speed and latency. However, it is difficult to represent the feature of the provided and various services from only the bandwidth and latency time in the integrated network environment.

In [19] the authors thought that pervasive computing environment brings the method of evidence context related to QoE. They studied the QoE evaluation method in pervasive computing environment, and proposed the enhanced QoE evaluation parameter model. In [19] rough-set based algorithm is proposed to reduce context attributes and determine the weight of each attribute, the algorithm has been validated on video streaming service, and the architecture of QoE evaluation system is described. As a mass of evidence information related to the experience of users can be gathered through the context-awareness computing, the calculation results of QoE evaluation method can highly match the real feeling of users. However, the method needs to be enhanced along with the development of pervasive computing.

Also, the existing research [16, 17] for QoE just deduced the major video quality elements of the IPTV service. They just analyzed a relationship of QoS parameters and QoE items and do not present a method for QoE measurement.

To solve the problems of previous studies, they have studied a model for measuring video quality at application layer using QoS quality parameters. In [15], we just showed the possibility to analyze the causes of the quality degradation through our idea of QoS/QoE correlation model. In previous works, however, we did not present the fair value of parameters, variables and constants in this model. Also, in [21], they need both the users and the provider/manager of the QoS system with a framework for the high level description of the system’s configuration and operation, as well as with an application/user specific definition of QoS. The feedback based quality assessment in best-effort network has some issues such as additional bandwidth consumption and network reliability assumption. Also, the feedback of the application layer and users is need for subjective QoE assessment. Separate quality management systems are inevitable to collect feedback information. However, our proposed model as a model for the formulation can simply evaluate the video quality satisfaction using only network QoS parameters.

2.3 QoE assessment model using QoS parameters

In [5], they present, demonstrate, and discuss a fundamental functional relationship between QoE and QoS parameters, the IQX hypothesis (exponential interdependency of quality of experience and quality of service). It is motivated and presented later. The use of such a QoE-QoS relationship is straightforward; by inserting measured QoS values into the corresponding exponential formula, their impact on QoE can be assessed immediately. Thus, the formulae presented and discussed in the sequel enable QoE threshold surveillance and control. However, Their QoE evaluation model is inadequate for evaluating QoE of multimedia service because it’s only considering the quality parameters of the voice service and web service. In addition, their model does not take into account the characteristics of the bulk arrival of streaming service packets.

In [20], they think that QoE parameters can be measured in the transport layer and the application layer of a TCP/IP network. Although there are many subjective parameters that can be included in the QoE, such as content availability, easiness and available content indexation, user interface, palette colors, ergonomics, navigation design and program guide, there are two main areas where the IPTV quality of experience can be measured objectively. Also, in [20], to define the QoEN parameter, they looked at the delay and jitter values and they saw that they have similar values in the networks where they performed our test bench. When they are high, the QoEN parameter should have a low value. On the other hand, although packet losses are very bad for the QoE, they could be zero, so that it cannot directly multiply the dividend. Higher values of packet losses affect the QoE value more; hence the e-number gives us the appropriate expression.

However, existing QoE evaluation models is impossible to measure the quality of the streaming services in real-time, because they need feedback from end-point to calculate QoE. Therefore, the video QoE assessment model which can assess video quality of experience using only measured values of the QoS parameters and their relative importance at network layer is needed for network and service providers.

3 Proposed video QoE assessment model

3.1 The QoS value normalization

As mentioned in the above, the user satisfaction about IPTV is under the influence of various QoS parameters. Thus, we limit a scope of QoE in this dissertation to the satisfaction about the video QoE of the IPTV service because overall QoE of the IPTV complexly consists of the variant QoE items.

It is known from the QoS/QoE relationship analysis that the most of QoE items are related with the QoS parameters. Moreover, according to the analysis results of QoS correlation, many QoS parameters show the positive correlation with QoE items. In order to reflect the variance of QoS quality parameters in QoE evaluation, the QoS value should be calculated. QoS value is a numerical data for normalizing network environment.

The QoS parameters having an effect on these QoE items are IP packet loss (L), IP packet delay (D), IP packet jitter (J), bandwidth (B), and etc. These QoS parameters are network-related quality elements which are recommended by the standardization organizations like ITU-T and IETF.

The weight of the QoS parameters should be assigned by considering the relative importance degree of the QoS parameter related to the IPTV video QoE and QoS/QoE correlation analysis results because QoE-related QoS parameters have the different influence. We can assign the weight of QoS parameters according to the quality standard bounds recommended in the standardization organizations (e.g. ITU-T, IETF etc.) and its relative importance degree as shown in Table 2.

Table 2 The analysis results of network-related QoS parameters of IPTV service
Full size table

The QoS value (QoS(X)) can be calculated through the formula (1), and reflects the network condition. In the formula (1), the constant K means the whole QoS quality determinant which is selected according to the type of the transmission system for the IPTV service. For example, we can assign 1 to K in unicasting.

$$ QoS(X)=K\left\{L\times Wl+J\times Wj+D\times Wd+B\times Wb\dots \right\} $$
(1)

The QoS value can be simply calculated with the total sum of the values multiplying the measured QoS parameter in network layer with the allocated weight like the formula (1). In the formula (1), it is just referred to the major QoS parameters influencing on the IPTV video QoE. The QoS parameter items used in this formula can be expanded and applied according to the characteristic of a service in case of considering the other multimedia services.

3.2 QoE assessment model for multimedia streaming service

Our previous research [15] only shows the possibility to evaluate the video QoS using QoS parameters. We verify the validity of the proposed QoE assessment model using QoS/QoE correlation through experiments using a variety of network environments and video samples. We devise QoS/QoE correlation model considering quality change aspect of MPQM according to network condition. The numerical formula model to measure the subscriber’s IPTV video QoE (QoEv) by using the normalized QoS value is as follows:

$$ QoEv= Qr\times {\left(1- QoS(X)\right)}^{\frac{ QoS(X)\times A}{R}} $$
(2)

where Qr is the upper bound of the video quality of experience according to the network type. Our experiments are carried out in the wired network, WLAN and WCDMA to determine Qr. Next, the QoS(X) is the QoS value which is calculated by the formula (1), and is determined by quality parameters of the network layer. The constant A expresses the subscribed service class such as SDTV and HDTV. If the subscribed service class is HDTV, the constant A is assigned in the higher value. It means that QoE level of the HDTV service subscriber is high than the SDTV service subscriber’s in the same network condition.

Equation (2) is a modification of the existing mathematical model (MPQM) to reflect the characteristics of the human visual system about the video quality changes. In the proposed model, the constants A and R are chosen as an approximation to express similar patterns and changes in the video quality measurement experiment. The proposed model can be elaborated through the experiments in various network structures and the type of service even if theoretical deduction is not enough.

The other constant R is determined as the constant reflecting the structure of the video frames according to the GoP (Group of Picture) length. Because an encoding is not progressed in case of losing an I-Frame until the next I-Frame is received, the loss of an I-Frame more induces the poor quality (frame skipping, frame freezing etc.) than the other frames (P/B-Frame). So, we can use the variable R as the factor for the video QoE measurement. A variable and the constant used in this model are determined by elements affecting video QoE. These elements include a service environment where the terminal is positioned, a service class, a used codec, and etc. In Fig. 1 we can know that change aspects of the QoE curve in accordance with service subscription class and GoP length.

Fig. 1
figure 1

Change aspects of the QoE curve in accordance with service subscription class and GoP length

Full size image

4 The experiment for justification of the proposed model

We configure experiment network like Fig. 2 to verify justification of the proposed model, and measure video quality using MSU video quality measurement tool [3]. And we use the VLC media player for video streaming and set up de-jitter buffer size to 100 ms. We convert XVID to H.264 codec using VLC’s transcoding functionality for the test which is widely used streaming format. Also, RTP protocol is used for streaming.

Fig. 2
figure 2

Experiment network configuration of video quality measurement for justification verification of our assessment model

Full size image

We use network emulation package called NISTNET [6] to analyze the relation video quality and network quality parameters such as delay, jitter and packet loss. NISTNET is network emulation software for Linux® that allows a Linux server running as a router to emulate a variety of network conditions, such as congestion loss, packet reordering, or asymmetric bandwidth conditions.

We compare 1,000 original video frames and 1,000 distorted video frames passed through experiment network. We do not use more video samples because it takes a lot of time in synchronization between original frames of the server and stored video frames on the client for video quality measurement. Table 3 shows the original video source information such as frame rate, resolution, codec and data rate for experiment.

Table 3 The original video source attributions such as frame rate, resolution, codec and data rate for experiment
Full size table

Network environment for test is configured for minimum required packet loss rate (0.1 %) and jitter (50 ms) which are recommended by ITU-T and IPTV standard organization. We consider only packet loss and jitter among various network QoS parameters related with video quality because they are the most influential parameters on video quality. Test is performed such that packet loss rate is between 0 % and 1 %, and jitter is max 60 ms with normal distribution to 200 ms delay. Quality degradation caused by end-to-end delay is not considered because we use the real time streaming rather than the VoD.

We use full-reference method to measure video quality like Fig. 3. Also, we choose SSIM (structural similarity) score to accurately measure video quality because SSIM is quite similar to MOS (Mean Opinion Score). Although SSIM is very successful model for representing still image quality, we can use the SSIM for video quality measurement because the video consists of successive still images [2, 25]. We can calculate the average SSIM for each frame by comparing the original video frames with distorted video frames. Here, the average SSIM can be used as an indicator of the video quality of experience.

Fig. 3
figure 3

Video quality of experience assessment by measuring average SSIM score

Full size image

Figures 4 and 5 show video quality deterioration with SSIM according to packet loss rate and jitter respectively. It shows that video quality deterioration to packet loss linearly changes while jitter’s sharply change around 50 ms. Video quality is uniform at high jitter because packet with high inter-arrival delay is dropped and video stream is normally played after receiving next key frame. Video quality more quickly deteriorates when packet loss and jitter are combined in the video quality.

Fig. 4
figure 4

Video quality deterioration changing patterns by packet loss

Full size image
Fig. 5
figure 5

Video quality deterioration changing patterns by jitter

Full size image

Figure 6 shows the example of aspect of video quality according to complex combination of QoS parameters. Although network providers offer network guaranteed packet loss and jitter within recommended QoS level, serious quality decline is caused when packet loss and jitter are combined like Fig. 6c.

Fig. 6
figure 6

The example of distorted video frames according to complex combination of QoS parameters

Full size image

Figure 7 shows relation between RSSI of a WLAN and mean video quality(MSSIM: Mean Structural Similarity). As see in the Fig. 7, data rate of video steams does not influence video quality when bandwidth is sufficiently provided. However, we show that video quality sharply decline when RSSI of a WLAN is lower than −80 dBm. Constant Qr in Eq. (2) is the upper bound of the video quality of experience according to the network type. Our experiments are carried out in the wired network, WLAN and WCDMA to determine Qr. The following Table 4 shows the results of the Qr determination.

Fig. 7
figure 7

Relation between RSSI of WLAN and mean video streaming quality(MSSIM)

Full size image
Table 4 Mean SSIM per energy consumption of mobile terminal comparisons
Full size table

Figure 8 shows the comparison of the proposed model and experimental results. The solid curve drawn by the proposed formula (2) represents the expected video QoE according to network conditions change. On the other hand, the dotted line shows the change in average SSIM of video used in this experiment. Through the analysis change patterns between network conditions and video quality, we assign constants A and R of the proposed formula (2) to 250 and 12 respectively.

Fig. 8
figure 8

The comparison of the proposed QoE assessment model and actual video quality measurement in various network environments

Full size image

As a result, we show that the video QoE measurement using the proposed model is very similar to actual measurement (correlation coefficient 0.937), and prove the validity of the proposed model. Therefore, we can simply measure video QoE in application layer using only network QoS parameters and the proposed video QoE assessment model.

Also, in Fig. 6, we can know that video quality of experience rapidly decreases when the normalized QoS value is less than 0.085. As shown in Fig. 6c, viewers can not recognized the image when SSIM is less than 0.85. So, we can infer that video service can be provided when the normalized QoS value is less than approximately 0.085. Therefore, network and service providers should maintain the normalized QoS value higher than 0.085 in order to provide seamless video stream used in experiment.

5 Conclusion

We propose the QoS/QoE correlation model to numerically measure the video QoE of IPTV by using the QoS parameters in network layer. Through our proposed model, network providers can predict subscriber’s QoE of video service in provided network environment, and derive the optimum service environment. On a real time basis, it is more rapidly able to correspond to the poor quality by monitoring the QoE of the IPTV service. The service provider can provide the multimedia service of the improved QoE through the proposed QoE control processes. Moreover, the network operator can prevent the unnecessary investment for the enlargement, maintenance and repair of the network.