Keywords

1 Introduction

To improve the performance of mobile network, many new approaches of network routing and measurement [1,2,3] are proposed. To internet service provider, the most important performance index is experience of user (QoE). Based on researches about user behavior and traffic analysis methods [4,5,6,7], new scheduling strategies are designed to raise resources utilization [8, 9] and improve QoE. With the rapid technology improvement and infrastructure deployment in wireless communication technologies including 5G (and later LTE-Advanced) and high-speed Wi-Fi, there have been a dramatic growth of mobile multimedia applications, for example, mobile video, 3D video stream, VoIP (video conferences), etc. [10]. These diverse content-rich multimedia applications lead to high complex traffic patterns and face user high requirements on QoE [11]. To measure the performance of these new scheduling strategies, flow level traffic reconstruction becomes an important topic [12, 13]. However, the inherent features of wireless communications, such as scarce bandwidth, interference, fading, error-prone channels, diverse access technologies and mobility, lead to a high level of dynamics of available communication resources that can deteriorate severely the quality of mobile multimedia applications with QoE constraints. There are bottlenecks for applying existed wireless techniques for ensuring wireless multimedia QoE. Such mismatch between the multimedia quality requirements and the service offered by the underlying communication infrastructure makes it a great challenge to develop mobile multimedia applications over wireless networks. Although the network operators and service providers make huge investments to improve the system availability, security and performance, mobile multimedia users still suffer from poor QoE frequently. Thus, new and efficient technologies are needed to improve the QoE for wireless multimedia applications.

As an efficient way to improve the wireless transmission efficiency, compared with traditional cellular-based homogeneous networks, heterogeneous wireless networks have attracted tremendous research efforts from both academia and industry over the past decade. However, most of the existing studies have mainly contributed to the performance enhancement of communication networks and paid less attention to the user-perceived QoE that desires the joint adaption of multimedia contents and the Quality-of-Service (QoS) enhancement of the underlying communication infrastructure. The optimal interaction between adaptive multimedia processing and heterogeneous wireless networking through a cross-layer design plays an important role in the efficient utilization of scarce wireless communication resources to balance the mismatch between the QoE requirements of mobile multimedia users and the QoS provisioning of underlying heterogeneous wireless networks with the aim of creating a new era where multimedia service providers, network operators and end users all benefit.

Due to the fast deploying mobile devices and surging amount of data exchange, the demands on better mobile applications and service qualities are always increasing. The conventional homogeneous framework could not be able to meet such requirements in a long run, especially when facing limited wireless resources and continuously emerging multimedia applications. Although several approaches, such as the densification of base stations, adding small cells and dynamic radio spectrum management, somewhat relieve the bandwidth pressure, they cannot effectively or efficiently solve the problem when more and more various communication demands emerge, from the perspective of service quality and costs.

Fig. 1.
figure 1

Topology of heterogeneous networks.

Full size image

On the other hand, heterogeneous networks provide a natural solution to boost the network capacity, to increase the network traffic and to enlarge the coverage. In contrast to homogeneous networks, heterogeneous networks consist of various types of transmission methods, radio access networks, and nodes with different power levels in the same network. The topology of heterogeneous networks can be illustrated in Fig. 1.

From the perspective of base station deployment, heterogeneous networks can be decomposed into two parts: one is the macro cells, which consist of high power base stations; another is low-power cells, such as Pico, Femto or Micro cells. Therefore, a heterogeneous network can also be treated as a combination of a macro cell and multiple small cells. More details on differences between the traditional cellular and heterogeneous networks can be found in [14]. Moreover, from the side of radio access technologies (RATs), the components in heterogeneous networks can be categorized into two classes [15]: one is single-RAT multitier components, and another is multi-RAT components. The former components are used in macro, pico, femto and micro cells, as well as the relays or client relays. The application situations for the latter group include WiFi offload, virtual carrier and mobile hotspots and personal area networks (PANs).

2 Challenging for QoE in Heterogeneous Networks

Lots of research efforts have been devoted to exploit the benefits of heterogeneous networks. Firstly, heterogeneous networks have the capability of addressing the problems induced by the rapid increase of network traffic. Secondly, heterogeneous networks are more energy-efficient since they allow communications with low power consumption. Thirdly, heterogeneous networks split the entire communication area into multiple cells with smaller size, which guarantees better QoS for user terminals. Finally, heterogeneous networks support a variety of radio access technologies such that the network performance has opportunity to be optimized with limited communication resources. A more detailed comparison can be found in Table 1.

Table 1. Comparison between traditional cellular networks and heterogeneous networks [14].
Full size table

Despite lots of merits by applying heterogeneous networks, there are still several challenges for using heterogeneous networks [15, 16]:

  1. (1)

    The cooperation is a crucial issue for heterogeneous networks. It is evident that, in heterogeneous networks, the compatibility of subsystems and interface standards determines their seamless interoperation.

  2. (2)

    From the technical point of view, since heterogeneous networks allow various types of low-power nodes, there will be sever imbalance in the utilization of limited resources, especially.

  3. (3)

    The heterogeneous networks can support far more user terminals than homogeneous networks and the designs of each subsystem can be compatibly different, therefore, the network infrastructure and radio link management will be in a very high complexity.

Despite high designing complexity, the heterogeneous networks exhibit great potentials in supporting rapidly emerging mobile applications, and satisfying the demands and expectations toward their service requirements from both service provider and end-user perspectives. This is especially true for an important form of mobile multimedia services, which will be introduced in the following subsection.

Benefiting from existing research efforts, QoE can be partly quantified, and its value can be predicted in some level by using objective methodology and subjective projection models. The heterogeneous networks scheduling strategies pose challenge to evaluate how scheduling affects the user experience.

3 Use Cases Analysis

Video, Audio and bursty data services are main scenarios in heterogeneous networks.

3.1 User Case 1, Video Streaming

Motivations

Currently, video is a dominant application in multi-media services. A recent report from Cisco shows that [17], without counting video exchanged through peer-to-peer (P2P) file sharing, the global consumer Internet video traffic will rise from 64% in 2014 to 80% of all consumer Internet traffic in 2019. It also forecasts that, the sum of all forms of video (TV, video on demand (VoD), Internet, and P2P) will be in the range of 80%–90% of global Internet traffic by 2019. For the mobile data traffic, it is predicted that [17], by 2019, global mobile data traffic will surpass 24.3 exabytes per month, and roughly 75% of the world’s mobile data traffic will be video by 2019. The results indicate that the mobile video service is acting a more and more important role globally. Up to now, the QoE assessment of video services has experienced four stages [18]: QoS monitoring, subjective test, objective quality model and data-driven analysis, and the comparison among those four assessment methods are given in Table 2.

Table 2. Comparison of video quality assessment methods [18].
Full size table

The video flow cost lot energy in network devices, the energy-efficiency is also important. Some new energy-efficiency strategies are proposed to save energy [19, 20]. It is necessary to evaluate how these strategies affect the QoE. The purpose of the evolution of assessment methodologies enables better QoE, and it also provides a motivation to investigate the mobile video service as well as its QoE in a systematic way.

Case Description

For streaming video in the Internet, one of most prevailing technologies is the HTTP adaptive streaming (HAS), which meets the growing consumer demands for mobile video services and enhances the QoE. Comparing with conventional streaming technologies, HAS has following four appealing and significant advantages [21]. Firstly, multiple rates of videos are provided, such that the delivered video can be adapted to the required standards by users. Secondly, different service levels and/or pricing schemes are available. Thirdly, the flexible service is offered to meet users demand on different kinds of streaming videos. Finally, the videos can be adapted to their best rate according to the current states of network and facilities in real-time. The last property, which can be interpreted as the reduced interruptions of the video playback and higher bandwidth utilization, is the most important one among all advantages over classic HTTP video streaming, and it contributes to the improvement of QoE of video streaming.

In HAS, a video is segmented into intervals that have durations between two and ten seconds, and each segment is encoded in multiple quality versions, where higher qualities correspond to higher rates, thus the number of segments equals to the number of rate versions. For HAS in wireless scenarios such as LTE network [22, 23], the progress is almost the same.Generally speaking, taking the QoE metric into account, the video transmission consists of four parts: encoder, networks, decoder and end users. In each part, there are factors that may finally degrade user experience. The typical visual distortions resulted by distorted videos are usually categorized by blocking effect, blurring, edginess and motion jerkiness, and those impacts seriously influence the QoE of video. As a subset of video service, HAS is also facing those factors that may deteriorate the QoE.

QoE of Use Case

For managing the point to multipoint multimedia communications in cellular networks, the Multimedia Broadcast Multicast Service (MBMS) is defined as a solution to deliver multicast and broadcast services over cellular networks by 3GPP [24]. The terminal devices can also provide feedback to the eNodeB using MBMS. From 3GPP Release 8, MBMS has been extended to the Long Term Evolution (LTE) standard. This extension is called evolved MBMS (eMBMS) [25]. To enhance QoE of users with degraded channels, the D2D resolution has been proposed for MBMS in [26]. In our opinion, MBMS should make full use of all available access networks to guarantee user experience. The most terminal devices using MBMS of cellular network support more access modes such D2D, Wi-Fi and blue teeth. We expect that the heterogonous network could improve QoE in the changing channel environment and save cellular communication resource.

In terms of performane measures, buffering time (startup delay), average PSNR and interruption percentage (rebuffering percentage) are proposed as three main QoE indices for MBMS in [27]. However, buffering time and interruption time/percentage are not strict QoE indices in psychological sense. We have to project this actual time to psychological time. Because no truly rigorous model can accurately describe the visual system, PSNR is still main index to evaluate user experience. Therefore, we have to check more indices for accurately describing user experience. For 2D video, the resolution, color rendition, motion portrayal, overall quality, and sharpness should be checked as important QoE indices. For 3D video QoE evaluation, the image quality, depth perception, naturalness, presence, and comfort degree are important indices closely related to user experience [28]. Although researchers have already achieved progress in the area of video QoE evaluation, we still meet some limitations and challenges for improving QoE of MBMS. The first problem is how to measure the effect of these perceptual indices. Although some standards have given the main human perceived features, it is still lacks the synthesis model to integrate these features. On the other hand, the user perceptual feature can also help efficiently use transport resource. For instance, based on the video resolution asymmetry between the left and right eye views, the main and mobile hybrid delivery for 3-D video services is proposed to maximize the channel efficiency in [29]. This means that the adaptive multimedia taking into account the user perceptual features, networking environments and media content could be efficient resolution for improving QoE of mobile multimedia services.

System Requirement of Use Case

We have to build more detailed index and explore the relationship between perceptual attributes of 2D/3D video and network performance index according to the objective and subjective evaluation standardizations recommended by Video Quality Expert Group (VQEG), ITU and European Broadcasting Union (EBU). In spite of network environment and service quality requirement, the adaptive multimedia systems should consider the user perceptual features of mobile video services. The locations of servers and clients should be also considered for access mode selection. This requires that the terminal devices should be able to gather and report the information related to QoE performance. This information is essential for the system adopting the adaptive scheduling scheme according to the current available channel resource, traffic state and QoE requirements.

To enhance the QoE, more and more Content Delivery Network servers are deployed on the core network border, or even in access network devices such as BSs. The proxy servers are deployed to reduce the handover frequency and support seamless handover for fast moving clients. Therefore, the test case should be able to set the moving mode of end users.

3.2 User Case 2, Wireless Audio Services

Motivations

Communications by voice is always a popular way for connecting people, since it is efficient and convenient. With the rapid development of wireless communication and significant proliferation of mobile devices, more powerful speech processing functions have been integrated into user facilities, such that a large variety of voice service, i.e. voice inquiry, voice remote control and audio conferencing, can be fulfilled to meet the ever-increasing needs in wireless and mobile scenarios. Those enhancement and effort make wireless voice service more attractive and popular.

As pointed out by, from the scenario point of view, the voice service can be classified into five categories: the PSTN voice service, voice services over IP (VoIP), hybrid voice service, and two other services that cross IP, interworking function and PSTN. Among all application scenarios, as Internet is playing an increasingly important role in modern society, the VoIP will see a more significant growth than its counterparts. Recent study and survey show that, over one third of enterprises now use VoIP and Session Initiation Protocol (SIP) trunking, and VoIP will take 66.51% of North America market in a few years [30]. For the sake of simplicity to discuss the topic, we narrow our scope to VoIP as a typical user case of wireless audio service.

For wireless VoIP service, clients care more about the network delay jitter [31], which can be regarded as a crucial QoE index. Furthermore, for some emerging intelligent interactive speech service, the edge computation strategies are also important for end user experience [32, 33]. The general metrics of QoE, instead of QoS, are also available for wireless VoIP service. Thus, there is a strong motivation to investigate its QoE performance.

Case Description

Generally, the variation of network determines the QoE of the VoIP service. The QoE of a VoIP service heavily relies on the network variation, such as the delay, jitter, and packet-loss [34]. Usually, the voice over Internet adopts the user datagram protocol (UDP), which can tolerate the dropping of several packets that cannot arrival at the receiver side before the deadline. It is known that, the transport protocol UDP is not reliable, and this kind of unreliability may yield severe degeneration of QoE of VoIP service. Regarding the mentioned problem, there are two classifications on the mechanisms to enhance the QoE of VoIP service, namely network-centric strategies and application-centric strategies [31]. The former utilizes a collection of compatible QoS mechanisms within the entire network to meet the requirements of services, such that the QoE of the VoIP service can be enhanced. For the latter strategies, the QoE is improved by optimizing the control mechanisms for end-users, such that the transmitter or receiver can adaptively cope with the voice data according to the network state.

By following the mechanisms and access technologies for wireless networks, it is easy to extend the conventional wired VoIP service to the wireless scenarios. An extensive application of such case can be referred to [35], where the SIP-based VoIP service in wireless mesh networks is investigated.

There have been lots of research efforts devoted to the investigation on network-centric and application-metric strategies, and their combination has gradually become a more popular tendency for improving the QoE of VoIP service. Such an approach is expected to be applied in a cross-layer design fashion in heterogeneous networks [36]. The merit of the combination is significant. On the one hand, all intermediate nodes in the network have the responsibility to monitor the variation of channel and conduct the appropriate actions to satisfy the requirements of QoS. On the other hand, applications behave adaptively to optimize the VoIP service. Both efforts improve the QoE of VoIP service.

QoE of Use Case

For the case of voice conversations, there are methods for QoE evaluation: subjective and objective speech quality assessment methodologies [37]. The subjective methodology gathers the perceptual feedbacks from users by using a collection of human subjects, to evaluate the QoE. Several subjective approaches are proposed to measure the quality of the degraded test speech with various objectives. However, many researcher found out that, the subjective methodology is time consuming and expensive, and they prefer to follow the objective methodology to overcome the drawbacks of its subjective counterpart. As stated in [37], one suitable objective speech quality assessment algorithm, which is closely related to intended goals and measurement context [38], can automatically, efficiently and reliably estimate the QoE. Furthermore, a game-theoretical method to manage the QoE of VoIP services in wireless scenarios is presented in [39], which explores a new vision to treat the QoE. From the user aspects, the QoE in wireless VoIP service is regarded as a function of the amount of efforts that the user has to put to preserve the conversation. These approaches need to be validated in well-designed test case.

System Requirement

To test QoE of audio service, the test case design must consider the characters of audio service in heterogeneous networks.

  1. (1)

    The test case must have different access mode with different capacity.

  2. (2)

    The capacity of different access mode should be dynamic, and the capacity can be observed.

  3. (3)

    The system should have the capability of detecting the wireless environment of client, since the environment might change frequently with time.

  4. (4)

    The QoE metrics should include latency and subject sense. The subject algorithm, such as P.862 and P.563, should be employed.

An Special Case: Mobile Voice Cloud

In addition to VoIP, we will briefly introduce another use case fitting the mobile audio services. This is called mobile voice cloud, which has recently attracted the attentions of both multimedia service providers and wireless network operators. The motivation behind mobile voice cloud services can be briefly described as follows. Due to the fact that normal mobile devices are small in size, the conventional way of inputting data to them (i.e., through typing buttons and/or pointing tools), requires sufficient attention and skills, and thus becomes inconvenient for many end users (e.g., those with poor eyesight, or attention drawn by other things such as driving). There is thus a need for intelligent user interfaces for supporting hand-free operation. Speech input (to computers) provides an obvious and promising solution for improved data entry/retrieval flexibility. There has also been a recent trend in interactive business and information applications such as call centers to move from a touch-tone based solution to a voice-touch driven approach. The voice input function can potentially be enhanced by the mobile voice cloud services. Specifically, upon receiving a voice request, the mobile client device sends the original voice data or speech features of the voice to the cloud service system. The cloud system recognizes the request and sends the response multimedia data to the users. Benefiting from this intelligent interaction system, users can therefore pay less attention to operate their mobile devices.

Architecture of the mobile voice cloud service can be illustrated in Fig. 2. The core technologies are voice and multimedia data transmission, distributed speech recognition, natural language processing, and information retrieval. The voice cloud service is a kind of request/response service. Therefore, the key performance indices for the user experience are response latency, success rate and the quality of response information. The adaptive transmission technology significantly affects response latency and the speech recognition accuracy, which determines the successful service rates. For improving the user experience in a changing environment, a cross-layer QoE-aware scheduling can be provided, by using network-aware adaptive multimedia processing at the application layer and multimedia-driven heterogeneous wireless networking at the network layer.

Fig. 2.
figure 2

Mobile voice cloud architecture.

Full size image

For these use cases, because the end users are moving fast. It is difficult to predict the network traffic flow. The traffic prediction method in end side is key factor affecting the user experience [40, 41].

After the multimedia-driven heterogeneous wireless network selection at the network layer, the network-aware adaptive multimedia processing will find a balance among different QoE indices such as response time and speech recognition success rate. This adaptive processing works in both of request phase and response phase. When the client assesses the available transmission capacity at the borderline of the request phase, the adaptive speech feature encoding scheme must find a balance between the accuracy of speech recognition and the transmission latency. At the response phase, multimedia content adaption methods also need to cope with the bandwidth-fluctuant radio channels and time-varying wireless communication conditions to achieve a better content quality and service profile that are suitable for the network environment. Multimedia coding transforms input media content into output content in a form that adapts to the channel conditions and meets the user’s needs. In the following, we will give some potential system design requirements that can actually improve the QoE performance of mobile voice cloud service:

In order to evaluate the QoE performance of very different services, one should build QoE index set for each type of service according to the way they affect the user experience.

The QoE evaluation model should consider both of the network parameters and service parameters. This model must be easy to guide the multimedia-driven heterogeneous wireless network selection and the network-aware adaptive multimedia processing crossing the application layer and the network layer.

A proxy for seamless handoff of voice cloud service in the heterogonous wireless network environment should also be used.

3.3 User Case 3, Mobile Bursty Data Service

Motivations

The mobile bursty data services, such as SNS, IM, OA, e-business, and so on, have been playing important roles in modern information exchanges. Yet, information exchanges among so many people generate the complex network traffic patterns. For instance, burst traffics often happen in access networks. Thus, single access mode cannot provide enough bandwidth and connection resource to prevent QoE deteriorating caused by traffic burst for all users. Since the smart terminal devices support simultaneously multiple access modes (e.g., cellular, WIFI), we need to study heterogonous network approaches, which may guarantee the user experience for mobile bursty data services.

Case Description

Currently, many people established and maintained real-time social connections with each other using bursty data services such as SNS, IM, Mobile OA, Mobile finance, in their mobile terminal devices. The characters of the transmission mechanism and devices determine how these services affect the user experience. On one hand, in general, these services adopt a round-trip transmission protocol in the application layer or the transmission layer. Therefore, the response time and the success rate of the whole round-trip transit process will affect the user experience in various ways. These services generally need to be constant online for use. Yet the high dropping rate on the move will certainly deteriorate the user experience. On the other hand, the short battery life has become a problem for mobile terminals for long time. From the point of user experience, the power consumption has to be considered in the QoE-driven heterogonous networks. The bursty data services usually have a relative long online time, even throughout the night, but they have a much shorter active time. Most traffic bursts on the networks between devices and access points are caused by a series of user actions, and the crowded people often caused traffic congestion in the backhaul of local access networks. User behavior analysis in application layer is therefore essential for the QoE improvement in the heterogonous networks.

Thus, the user actions and events trigger data stream in bursty data services. The data transmission qualities of some actions affect the user experience. According to a rough classification in reference [42], bursty data applications belong to transactions-oriented applications characterized by request/response data flow corresponding to bidirectional data transfers. Thus the user experience is mainly related to the delay of the answer to a request. ITU-T has proposed the criterion for web-based service QoE evaluation [43]. Although the response latency is still the most important QoE index for bursty data services, most bursty data services cannot be referred simply as web browsing. Usually, the user can only perceive the quality of whole transmission process triggered by one action. This requires that the scheduling scheme must be able to find the scope of the human-perceived user actions in a number of transmission layer packets. In spite of waiting time, the long delay might causes high dropping rates that also deteriorate the user experience and leads to a disconnection, which may confuse to the users using SNS or IM, even if users cannot directly perceive the delay.

QoE of Use Case

The latency and success rate of human perceived round-trip processes in the services are closely related to user experience. We split the round-trip process of bursty data services into two classes. (1) The process can be perceived by users. The delay and success probabilities of some actions can be perceived by users, e.g., the delay of publishing a short video in web site, and that of submitting a comment on blog. (2) The process cannot be perceived by users. Although the delay of some actions cannot be directly perceived by users, a too long delay will cause confusion to users. For instance, at the chat window of IM services, users cannot perceive the latency of sending messages. However, long delay might cause the messages to arrive at the receivers in wrong orders and thus lead to a terrible logical mess to the users. Note that since the high dropping rate might lead to negative experience, the round-trip communication for keeping constant online also might influence the user experience. Moreover, the energy consumption of service might influence the user experience. There are two key principles for identifying these human-perceived round-trip processes. (1) The whole human-perceived round-trip process should be identified as a basic element in QoE-driven scheduling. This process usually includes several round-trip communication processes in the application layer and network layer, which cannot be perceived by users. (2) Some processes deteriorate QoE with long response time, and some cause negative experience with failed communications. There are different concerns for different types of processes. Moreover, the different type of action affects the user behavior in different way. For instance, some actions are delay-sensitive and some actions can tolerate relatively longer delay. Therefore, the projection models from actual time to psychological time should be very different. Since the high discharge rate is also influence the user experience, the well-designed relationship model among battery life, service energy consumption and user expectation is also necessary for QoE-driven heterogonous network design. Because most bursty data services have their own proprietary application layer protocols between server and client, understanding those proprietary protocols is another key problem to identify the type of action and to adopt adaptive KPI-QoE projection model in the heterogonous network design.

Test Requirement of Cases and the Concerns

The key of QoE-driven heterogonous network design is to identify user perceived index. Although some index sets have been build for web-based services [44,45,46], they cannot cover the burst data services. Only a few researches focused on the relationship model between these indices and QoE.

The user behavior is important for design bursty data service test case. Because of the complexity of user behaviors for bursty data services, it is not appropriate to validate the QoE-driven heterogonous network design in single user simple behavior scenarios. The test bed should use real data to validate the ability of scheme to identify service type and adopt corresponding strategy. Conventional testbeds usually activate data flow and build scenarios according to the traffic model. The traffic model usually characterizes the fluctuation of overall data flow [47, 48]. It is difficult to analyze the relationship between user behavior and user experience. The most traditional test systems send random data so that they cannot test the ability of service type identification of scheduling scheme.

According to the challenges for QoE improvement, heterogonous network design and system validation, the system requirements are described as follows:

  1. (1)

    We have to build the user perceived index sets for each popular burst data service type, and investigate the connections between the indices and user experience.

  2. (2)

    A test-bed for burst data services should be able to assemble the different scenarios based on understanding the user behavior in real world, and evaluate the performance index according to different service type. The testing process is illustrated in Fig. 3.

  3. (3)

    The test-bed must be able to replay and analyze real data packet.

  4. (4)

    We have to build a typical user behavior warehouse for assemble different scenarios based on user behavior analysis in real world.

  5. (5)

    The test system has to measure the traffic via intelligent algorithms, and build test cases based on the traffic scenarios [49, 50].

Fig. 3.
figure 3

Structure of the test-bed for burst data services.

Full size image

4 Conclusion

In the document, we summarize the recent development on the multimedia application in heterogeneous networks. Especially, we discuss the recent development on Quality of experience of wireless multimedia applications. A brief introduction on heterogeneous networks, mobile service, QoE and their analysis method is given. For different kinds of services, the requirements of test cases are proposed based on the use case analysis.