Context-Aware Content Adaptation for Personalised Social Media Access

Abstract

Accessing, sharing, and delivering social media involve online user-to-user social interactions. These interactions can be characterised by one-to-many or many-to-many communications established amongst numerous users. This in turn makes the whole environment a very heterogeneous one, comprising diverse user devices, fixed and mobile access network technologies, content representation formats, user needs and preferences, and media usage and consumption environment characteristics. Owing to this very heterogeneous nature of social media access, personalised access to the social media is a significant challenge for maximising the user experience and satisfaction. This chapter presents context-aware social media content adaptation as the key technology to address this challenge. It introduces the importance of context awareness in personalised social media access, and how it can be coupled with content adaptation and adaptation decision-taking mechanisms to provide a complete solution for both the technical and non-technical (i.e. social) challenges faced. Detailed discussions focus on describing various adaptation and decision-taking types and operations while also pointing at a number of open issues for future research, so as to address those highlighted challenges for realising true personalised social media access environments.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

In the context of networked digital media delivery, personalised access to multimedia content embodies all of the necessary strategies that are needed for effectively responding to the ever-changing and increasing preferences of users of media services. These strategies encompass the development of several key technologies, most notably content adaptation, for providing the users with what they would like to receive, watch, see, hear, and read, in the way that they prefer.

In an era, in which content has fast become ubiquitous for consumption, personalised access to content and content services has also become extremely demanding with the availability of many media networking and access technologies as well as fancy and affordable user devices. Challenges arise due to both technical and social reasons. Technically, the vast variety of access network technologies, content types and formats, and terminals have led to tremendously heterogeneous digital media distribution and consumption environments. Social challenges have been created as a result of users’ various demands, preferences, and expectations from the services that are provided. In turn, both render access to content very challenging. Particularly, with the proliferation of the recent social networking and content sharing facilities, these challenges have been exacerbated, as both the number and demands of users have significantly increased and varied.

Nowadays, there seem to be a large number of social networks and thus networked communities of users active over the Internet for every need or reason, which promote communication of people and sharing of content with each other, regardless of their diverse age groups, backgrounds, locations, special requirements, preferences, etc. The growth in the amount of user interest in social networks has made content sharing a part of the daily life. Consequently, there is an increasing tendency of sharing more and more media content without paying much attention to the diverse content types produced and shared, the user devices used to capture, generate, and consume those various content types, and the communication links that are used as the access networking backbone with their respective limitations.

In light of the above discussions, this chapter proposes to provide in-depth knowledge on how context-aware content adaptation, as a key technology, is able to constitute an efficient strategy to address those aforementioned challenges for personalised social media access. Therefore, this chapter first provides a close look at the context, which drives the content adaptation. Often, context imposes constraints and requirements for the information to be delivered to a given user or a set of users. For instance, available network bandwidth limits the data rate of a video delivered to a user. Since maintaining the optimum user perception towards the consumed media is a key requirement, accurate sensing and understanding of the contextual information plays a significant role in efficient social media sharing and delivery applications. Hence, context sensing and classification technologies are introduced, and context in social media consumption environments is analysed in detail.

The heart of a content adaptation system is the signal processing techniques, which perform the actual adaptation. The signal processing techniques used to make the input media suitable for a content consumption environment, which is described by the contextual information, are investigated in this chapter. Typically, adaptation mechanisms can be located anywhere in the end-to-end media delivery chain. In terms of their functionality, these mechanisms can perform a variety of content adaptation operations and can be classified in terms of level of adaptation applied on the content. Some of these adaptation mechanisms perform signal level transforms, such as transcoding, transrating, and transmoding. Others may apply semantic level transforms, such as cropping and highlighting of audiovisual attention areas, while another group of adaptation mechanisms transforms the input signal at structural level, such as summarising and mosaicing a lengthy video stream. Besides the abovementioned classification, the adaptation mechanisms can further be categorised in terms of whether they are executed offline or online, and also depending on the purpose of adaptation. These classification strategies are discussed in detail in this chapter. Subsequently, in-depth discussion of the signal processing techniques and tools, which are most applicable for social media applications, are also presented.

Once the contextual information is obtained, choosing the most efficient and effective signal processing technique from a large number of available techniques for adaptation is a challenging issue. This is the exact role of an adaptation decision-taking algorithm. Due to the cognitive nature of its functionality, adaptation decision taking is considered as the brain of a content adaptation system. This chapter presents a comprehensive discussion on the adaptation decision-taking algorithms. First, context reasoning, which involves processing of raw contextual information to unearth constraints and requirements that solicit the need for adapting the media, is explored. Subsequently, two adaptation decision-taking approaches, namely knowledge-based and utility-based adaptation decision taking, are elaborated.

Social media adaptation is not always as straightforward as it is anticipated in a user-to-user communication scenario. There are many technological and non-technological (i.e. social) issues that may inhibit or discourage performing content adaptation. Thus, last but not least, this chapter elaborates on those challenges with necessary pointers to open issues and potential directions for addressing them efficiently. Finally, this chapter is summarised and concluded.

1.1 Ambient Intelligence for Personalised Social Media Access

In the beginning of the pervasive social networking age, like many earlier multimedia services, social media access, sharing, and distribution were also limited to mostly home and office environments with devices connected to fixed networks supporting broadband links. This made the provision of such services rather easy and manageable, as dedicated links and terminals were targeted while providing social networking and content sharing with various online communities. This also meant that the intelligence for providing device and/or network centric media content customisation capabilities during online social activities mainly lay centrally within the networking infrastructures for supporting those services with acceptable quality of service (QoS) levels. Nevertheless, with changing times and fast proliferation of ubiquitous computing and communication systems, a great number (if not all) of the media services that we used to receive only in our homes have migrated rapidly to the mobile environments. Without doubt, this has led to a wider uptake of social media access and content sharing with even wider communities, who were not fully foreseen prior to such migration.

Thus, not only has this resulted in the need to cater for a wide range of terminals (both fixed and mobile) and networks (both wired and wireless), as was the main focus of the past media services, but also accelerated the research and technology development activities for placing users themselves in the spotlight as the real consumers of media content at last. In turn, personalisation aspects have gained significant weight for accessing and sharing the social media while paving the path towards enhancing the quality of experience (QoE) of users of those services. Evidently, this has also presented an alternative to the centralised approach through realising distributed intelligence within the networked digital media delivery chain to provide personalised social media access.

Distributed intelligence starts from the surroundings of the user, which can influence the access and delivery of social media content and its successful sharing, consumption, and wide-scale acceptance by all parties that take part in an online social interaction. This concept is widely referred to as ambient intelligence in the computing literature [1, 2], where electronic environments that are sensitive and responsive to the presence of people (in our case, social media users) are addressed. This is a natural evolution of ubiquitous computing and communication systems that are enabled by the smart equipment and devices with sensory elements as well as intelligent networks and networking peripherals. The intelligence is embedded within the systems that are context aware, where personalised and adaptive services can be supported with an anticipated level of user satisfaction.

In personalised access to social media, ambient intelligence thus plays a very significant role on the successful user centric delivery and consumption of content towards enhancing the user QoE, which is easily affected by the user preferences. The core of ambient intelligence relies on the sensed context in the environment. In fixed environments, the ambient effects can be predicted (if not measured) accurately, as these effects generally present measurable and predetermined variations. On the contrary, mobile environments have an ever-changing nature, which demands continuous monitoring and sensing of the changing context. To exemplify this argument simply, one can consider the differences in access to a social networking and content sharing platform using public and private fixed user devices. As two different devices are fixed to either public or private context, based on where they are located, a dedicated protocol can easily be followed for access granting and content sharing in both cases. However, while using a mobile smart device, the boundaries for public and private access may become blurry when the user location frequently changes. Therefore, this situation calls for regular context updates to sense and follow the variations, so as to be responsive to the new conditions accordingly.

Interaction with the ambient intelligence within the environment can be achieved both automatically, where devices and computing elements react autonomously, and semiautomatically, where user interacts with the execution of the protocols and algorithms through various user interfaces. This is particularly important, where user preferences can be provided as a live and accurate feedback to a content adaptation system that serves the user for personalised social media access and consumption.

Further intelligence that supports personalised access to social media content lies in the architectures of servers, access networks, and terminals in an end-to-end media access, sharing, and delivery chain. Such intelligence is provided by employing necessary media content adaptation capabilities at those different points within the delivery chain, which will be further introduced and discussed in the subsequent sections.

1.2 Need for Content Adaptation in Social Media Access

In a nutshell in the context of this chapter, the term social media refers to the rich multimedia content that is exchanged, shared, consumed, and enjoyed over the Internet socially. Since access, sharing, and delivery of social media involve an online social interaction, they also result in connecting a very wide range of users, who may or may not know each other physically, in one-to-many or many-to-many communication modes. Such diversity means a heterogeneous media access, sharing, and delivery environment, which comprises different user devices, and access network technologies, as well as various content representation formats, user preferences and ‘intentions’, and media usage and consumption environment characteristics.

The success and wide proliferation of the Internet and mobile communication systems together have motivated the development of various enhanced-capacity fixed and wireless networking technologies (e.g. 3G, LTE, WLAN, WiMAX, broadband Internet) [3]. This has encouraged growth in the variety of formats with which media content can be represented for various media applications and services, including social media access and sharing. These services supported by such networks helped to foster the vision of being connected at anywhere, anytime, and with any device particularly for pervasive social media applications. However, the coexistence of the different networking infrastructures and services has also led to an increased heterogeneity of compressed media communication/delivery systems and application scenarios, in which a wide range of user terminals with various capabilities access rich media content over a multitude of access networks with different characteristics.

In a typical social media access scenario, content may be captured, preprocessed, and prepared for exchanging and sharing using a particular format, which may be specific to a capturing device (e.g. a high-quality handy-cam). In addition, the user, who shares this content with his/her peers in a social networking platform, may wish to draw the attention to specific parts of the content through either manual or automatic editing and metadata tagging. The device is connected to an access network or a bundle of networks directly or through another networked device (e.g. a multimedia tablet or desktop PC) for transmitting the information with their respective requirements, such as bandwidth limitations, delay, jitter, and error performances, etc. All of these pose non-trivial challenges for sharing this social media content with the intended level of QoE, as the other users may or may not be able to match their terminal capabilities, access network characteristics, or content formats for accurate processing, rendering, displaying, and presentation. Furthermore, their preferences may not adequately mirror those of the content creator’s/producer’s, as for instance they may wish to focus their attention to other portions of the content where it is more attractive to their taste.

As social media is an intensely user centric concept, it is also important to cater for a mechanism to provide user feedback into the content access and sharing platform. This necessitates the consideration of various modes of user interaction, again which may not have similarities with the modes used for preparing the content in the first instance, may not comply with the specifications set by the content producer or may have specific requirements due to content usage environment. The environment, in which the content is acquired and consumed, may also impose changes in the way the content is rendered and presented in contrast to the originally captured and shared content, as previously discussed. All of the aforementioned mismatches may thus hinder a guaranteed or acceptable level of QoS as well as QoE of the social media users.

The mismatches between the media content properties and several network and/or device-centric features, usage environment characteristics, as well as diverse user preferences call for efficient content delivery and sharing systems featuring effective context-aware content adaptation mechanisms for personalised social media access. In general, this concept has been widely addressed in literature with the theme of universal multimedia access (UMA) [4, 5]. The big picture addressing the need for content adaptation in the UMA concept is depicted in Fig. 1.

Fig. 1

The big picture of the UMA concept and need for content adaptation

1.3 Core Essentials of Content Adaptation

Content adaptation refers to the set of mechanisms and technologies to provide effective solutions for meeting various mismatches between media content properties, diverse access network and device centric features, user preferences, and usage environment characteristics in a social media access and content sharing scenario. The overall aim here is to satisfy a set of technology imposed constraints for seamless and personalised social media content access, sharing, and distribution while improving user experience in terms of perceived media quality and satisfaction from the delivered social media services. Thus, content adaptation is the process of transforming an input media stream to an output media or augmented media representation format by manipulating the input media signal at different levels (signal, structural, or semantic) in order to meet diverse resource constraints and user preferences while optimising the overall utility and usability of the media content.

Three core components determine the operational success of content adaptation: context awareness, actual media adaptation, and adaptation decision taking. An adaptation system should be aware of the contextual information that characterises the social media access, sharing, delivery, and consumption environment. The features of the user device that is used to access the social media content, the characteristics of the access network that the device is connected to, user’s location, content access time, the ambient conditions of the environment where content is accessed or shared, etc. are a few of the key contextual drivers that influence the way the content can be used. Through accurate sensing and monitoring these contextual elements, the selection and working of content adaptation methods can be made responsive to the operational specifications of the content access and sharing platform adequately. Actual media adaptation involves applying a suite of signal processing techniques on the input media signals, so as to minimise the effects of the aforementioned mismatches between content properties and usage environment conditions with the help of contextual information that is collected and inferred. All of this process is overseen by an adaptation decision-taking mechanism, which constitutes the intelligent component of a content adaptation system. Given a context, there can be a number of media adaptation methods that can be applied, which achieve similar results. Nevertheless, the decision to select and apply the most effective method is the main responsibility of the adaptation decision-taking component. The core essentials for performing the adaptation of social media for personalised access are discussed in the following sections of this chapter in detail.

1.4 Middleware Architecture for Social Media Adaptation

Distributed intelligence for handling content processing to provide personalised social media access within the networked digital media delivery chain requires a distributed approach to social media adaptation. Social networks are the prime examples of person-to-person or in other words user-to-user communication and media sharing platforms. In such a networking platform, users can act both as content producers and consumers, where the focus is on assistive technologies that are aimed to adapt to the heterogeneous usage environments rather than users trying to adapt to technology specifications. Thus, once content is shared with peers in online social communities, it is not the main concern of the content producer to ensure the shared social media is accessed and delivered to all parties at their preferred levels of user experience or in compliance with their service level requirements adequately.

Indeed, in most cases, content (captured and prepared by professionals or lay users alike) is uploaded to a content sharing platform and expected that it will be accessed and downloaded by every user with similar experience successfully. Nevertheless, it is not always the case in reality, as a high-definition (HD) resolution media content uploaded using a high-broadband Internet connection for sharing with others cannot be easily accessed and downloaded by users in full resolution or fidelity over a mobile wireless network (e.g. 3G) with limited bandwidth using a smart phone with a display size at a fraction of the HD resolution and low-capacity processing power.

Thus, social media adaptation is needed, so that all users in an online social circle can access the same content at varying fidelities that suit their needs and preferences as well as their technology requirements. Adaptation is an application layer middleware operation, which sits between content coding and decoding acting as a processing tool. The strategic positioning of the adaptation and decision mechanisms in a social media delivery platform is the determining factor for the success and effectiveness of content adaptation for personalised social media access. Three middleware architectures have been widely reported in literature based on this strategic positioning: server, network, and terminal-side architectures, as illustrated in Fig. 2. These architectures have their respective advantages and disadvantages associated to the factors (e.g. content processing load, transmission bandwidth need) that affect adaptation operations that can be performed [6–8].

Fig. 2

Location/architecture of adaptation. (a) Server side middleware architecture, (b) Network side middleware architecture, (c) Terminal side middleware architecture

In server-side architecture, content is adapted at the main content server level, where a number of copies of the content at different resolutions and/or qualities can be prepared for distribution when needed [8, 9]. Similarly, content specific preprocessing, scalable layering, error resilience addition, or multiple description-based encoding can be applied at the server for targeting heterogeneous communication and content delivery networks. The server is the sole responsible for gathering the user preferences and requests, device capabilities, and the available bandwidth on the network, performing adaptation, and sending the adapted content to the user terminal in this architecture. The main advantage is that it allows both offline and online media adaptation for social media access, while also providing the author of the social media content with tools for better controlling the operation and presentation of the shared media. The main disadvantage is the computational load on the server, particularly for offline adaptations.

Content can also be adapted using a network-side middleware architecture, which can be located within the social media access, sharing, and delivery platform. This point can be referred to as a gateway or proxy node on the edges of heterogeneous networks, so as to match mainly network centric diversities that significantly influence both the QoS and QoE of the received media at the user terminal end. Device capabilities can also be addressed in this kind of an adaptation mechanism. The adaptation requests are sent to the relevant proxy or gateway rather than all the way to the content server. When the proxy/gateway receives the request, it acquires the adequate context from the user and his/her terminal, and it connects to the server on behalf of the user/terminal to gather the adequate content information. Then, the proxy/gateway decides on the adaptation operation, executes it, and sends the adapted media to the user. Thus, at such network points, the specifics of the social media content are mainly adapted based on the characteristics of the next host transmission medium for service level requirements, such as access bandwidth matching and error robustness insertion, as well as spatio-temporal scalability of the content can be provided for addressing various terminals in the usage environment. This type of architecture is advantageous in terms of bandwidth, as it takes the advantage of the bandwidth between the server and proxy or gateway, which is relatively high in most cases [8, 10]. The network-side architecture alleviates the disadvantages of the server and terminal-side middleware architectures [7].

Media adaptation can also be performed at the terminal end depending on the device capabilities in the terminal-side architecture. This architecture has a disadvantage that it can be sometimes impossible to perform adaptation due to the limited processing power of user devices, such as PDAs and mobile phones [8, 11]. Moreover, sending high-quality media content to the user terminal and leaving the adaptation to the device may cause high network traffic all the way throughout the social media access, sharing, and delivery chain [7]. However, this kind of architecture also has its advantages, such that adaptation on the user terminal targets accurate rendering and presentation of the content, while also providing a valuable means for better personalised content access taking into account the user preferences. Preferences can be easily input to the content adaptation system through simple user interaction modes supported on the user terminal.

In a typical media access, sharing, and delivery scenario, it is not necessary having to choose and abide by one point of adaptation, as performing distributed adaptation at a combination of various adaptation locations may result in enhanced quality media content at the receiving end. Media adaptation can be applied in a cooperative fashion, such that at all three strategic adaptation points, a set of different requirements and constraints can be addressed by employing accurately selected media adaptation method and middleware architecture. This can be achieved by applying the adaptation decision-taking mechanism also in a distributed fashion to closely monitor, supervise, and drive the adaptation operations to be performed at different positions in line with the contextual information collected at those respective locations. In this way, distributed media adaptation architecture can be realised, which aids in distributing and balancing the adaptation load across servers, network proxies/gateways, and terminals adequately [7, 12].

2 Context Awareness in Media Networking

According to [13], a system is context aware if it uses context to provide relevant information and/or services to the user. Context awareness refers to the state of being aware of the context in a sensed environment [14]. As previously introduced, it is one of the core essentials for performing media adaptation to address context-related constraints and requirements in a digital media access, sharing, and delivery environment successfully and effectively. Context-aware systems have thus ability to sense, infer, and react to contextual information, such as network conditions, terminal capabilities, natural surrounding environment characteristics, and user preferences, by adapting their behaviour dynamically [15]. The use of contextual information to facilitate decision taking on how to adapt media content based on that information is key for implementing meaningful media adaptation operations that meet users’ expectations while also satisfying their usage environment constraints.

2.1 Definition of Context

If a user and application are in interaction, any information sensed at the time of this interaction can be identified as context [16, 17]. Context has numerous definitions in literature depending on the context-aware application it is used in [18]. The most generic definition is given in [19], where context has been defined as the information to characterise the situation of person, place, and object, which are in interaction with each other.

Thus, contextual information can be any kind of information that characterises or provides additional information regarding any feature or condition of a complete content delivery and consumption environment. Use of context is particularly important for deciding adequate adaptation parameters that are needed in adaptation operations. Contextual information can be exploited in content adaptation systems as metadata, which is required for driving the adaptation decision-taking algorithms to determine adequate content adaptation methods in response to the constraints imposed by the context of usage.

Contextual metadata used in the decision algorithms describes the characteristics and conditions of the context of usage for networks (e.g. bandwidth availability, error rates, jitter), user terminals (e.g. screen size, CPU, codec capability), natural surrounding environment (e.g. ambient illumination conditions, noise level), and users (e.g. disabilities such as hard of hearing or colour perception defects, preferences such as language, summary of images versus video, a specific view in multi-view content).

2.2 Context Gathering and Classification

Context-aware systems need to acquire the contextual information, so as to process and reason about this information to further formulate concepts and take decisions when and how best to react to it. Gathering contextual information takes place in three steps: sensing low-level context, inferring high-level context, and sensing changes in the context. The first step relates to the generation and representation of basic contextual information, as it can be directly generated by a software or hardware application. In most cases, low-level context can thus be acquired through the use of physical tools, such as sensors placed on user devices or in the natural environment surrounding the user. Automatically generated data from software modules also allows collecting necessary low-level contextual information concerning terminal capabilities and network characteristics. On the other hand, information addressing the user preferences and his/her natural surrounding environment requires the use of dedicated hardware as well as occasional (or sometimes even regular) manual intervention from the user. Dedicated hardware includes visual and aural sensors, such as video cameras and microphones, as well as other user terminal interfaces. User preferences can be implicitly created based on usage history, but most often they require the explicit inputting by the user, at least during system start-up.

Based on the basic (or low-level) contextual information, applications may infer higher-level concepts in the second step. For instance, emotions of a user or his/her physical state can clearly affect his/her preferences about a media content or service, particularly in social media applications. This information can be inferred by analysing low-level contextual information obtained by imaging, sound, or medical sensors. Similarly, location data may be collected as low-level information using geographical coordinates during the context gathering stage. When this data is analysed for inferring the associated high-level context, the type of physical space the user is in can be determined, such as indoors, outdoors, train station, bus stop, sports stadium, etc.

Once low-level context is acquired, and high-level context has been inferred from it, the third step is to monitor and sense the changes that occur in the contextual information. Therefore, the acquisition of context, at least of some types of contextual information, should be a continuous process, regardless of the fact that it is a periodic process or not. In this way, changes in the context can be perceived by the context-aware application. Evidently, this necessitates the reasoning about the basic contextual information to be a continuous process, which is conducted whenever changes in the basic contextual information are detected.

Typically, context can be classified into four classes in a context-aware system [18]:

• Resources context

  • Description of terminals in terms of hardware and software (available processor, battery, screen size, operating system, codecs, etc.)

  • Description of networks (available bandwidth, maximum capacity, bit errors, packet losses, etc.)

  • Description of multimedia servers (maximum number of users, maximum throughput, etc.)

  • Description of transcoding engines in terms of their hardware and software (input/output formats allowed, bit rate range supported, etc.)

• User context

  • Description of user’s general characteristics (gender, age, nationality, etc.)

  • Description of user’s preferences in terms of content consumed and his/her interests (type or language of the media preferred, action movies versus comedy, etc.)

  • Description of user’s emotions (anxious, happy, sad, etc.)

  • Description of user’s status (walking, talking, etc.)

  • Description of history/log of actions performed by the user

• Physical context

  • Description of environmental information surrounding the user (location, temperature, ambient illumination conditions, sound/noise levels, etc.)

• Time context

  • Description of time at which the context is measured, variations in the context have occurred or scheduling of future events (time of the day, frequency, etc.)

Besides, there are a few other factors that affect the contextual information that is gathered, which can be summarised as follows:

• Accuracy or level of confidence of the contextual information

• Validity period

  • Contextual information may be static, thus having an unlimited validity period (e.g. general characteristics of a user are static and do not require any additional information to be used), or it may be dynamic and valid for only a given period of time (for instance, user’s emotions and conditions of the natural environment, such as illumination or background noise, are dynamic).

• Dependencies on other types of contextual information

  • Reasoning about one type of contextual information may depend on other types of contextual information.

2.3 Use of Context in Social Media Adaptation

In a typical end-to-end social media access, sharing, and delivery platform, contextual information can be obtained by the following available context providers:

• Users: via their terminal devices

• Network operators: via network equipment, including network probes

• Content providers: via streaming servers, encoders, databases, etc.

• Terminal equipment vendors: via a variety of supported device specific sensors

• Natural surrounding environment: via user terminals and other environmental sensors

Contextual information, once acquired with the help of the aforementioned context providers, is used to exert constraints and requirements on the social media that is accessed, shared, and delivered. Context use in media adaptation is illustrated in Fig. 3. Context is usually fed into the media adaptation mechanisms in the decision-taking phase in the form of Universal Environment Description (UED) and Universal Constraints Description (UCD) tools, as described by Part-7 of the MPEG-21 Standard on Digital Item Adaptation [20–22]. UEDs are used for describing the environment, in which the media content is transmitted, stored, used, and consumed. The necessary information for providing the UEDs comes from contextual data. They are divided into four main groups: user characteristics, terminal capabilities, network characteristics, and natural (usage) environment conditions. Together with UEDs, UCDs are also used in adaptation decision taking. They provide the description of limitations and optimisation constraints on possible adaptation operations. This tool enables users to further constrain the usage of media content; for instance, the resolution of the rendering device can be constrained to satisfy needs of the application or device display size using the UCDs. Social media providers may also restrict the way their content is adapted and used, such as imposing a minimum level of quality or user profile during consumption by help of this tool, which provides a control over the adaptation operations performed upon the content. Constraints that are described by UCDs can be derived from UEDs and can also be used to further constrain them.

Fig. 3

Context use in media content adaptation

Processing and analysing a set of low-level context elements gathered with the help of context providers allow inferring high-level contextual information in a social media access, sharing, and delivery scenario. For example, natural environment-related contextual data may lead to the understanding of whether the social media is accessed and consumed by the user in a private or public location (e.g. at home or work place). In turn, this knowledge helps the social media adaptation system make necessary personalised adaptations according to the situation either by allowing the required adaptation or offering an alternative solution. A mobile media consumption environment inferred through relevant context providers results in adaptation of social media content shared by the content provider to the mobile and wireless context. This may include network bandwidth and device display size matching, user preferences-based content summarisation where full version of content cannot be processed or viewed on a smaller user terminal, etc.

Together with all of the context elements, content-related metadata also enables taking better decisions for personalised social media access. This is particularly desirable in social networking applications, as content is usually shared with all online communities without explicit knowledge of who is actually receiving the content. It is very common that most of the social media content shared in social networks is not appropriate for all users. Therefore, based on the content metadata, either added by the content provider manually or extracted and embedded within the media stream automatically, social media content adaptation can be performed to suit the preferences and needs of a diverse range of users, who may have different backgrounds, requirements, age ranges, understandings of the available content, etc.

3 Signal Processing Techniques for Content Adaptation

Any adaptation operation that transforms input media into a target form involves signal processing. Often, this is the most computationally expensive stage of the entire media adaptation operation. Signal processing tools can be deployed at any point in an end-to-end social media access, sharing, and delivery chain. Some of the tools intercept media even before they leave the server or producer while other operations are performed at the user terminal or in the communication network, as was illustrated in Fig. 2. Figure 4 shows a few of the example signal processing technologies that can be deployed to perform social media adaptation in the end-to-end communication chain.

Fig. 4

Typical social media adaptation examples

Signal processing mechanisms available for social media personalisation can be classified in a number of ways. Some of these mechanisms perform signal level transforms. This type of transforms preserves key information enclosed in the source media as much as possible while transforming the media into the target format. However, the quality of the media content may degrade during the transform. Transcoding, transrating, and transmoding are a few examples of signal level adaptations [23, 24]. This group of signal processing mechanisms is the most common adaptation operations that can be used for personalising social media. Another type of signal processing mechanisms performs semantic level transforms, such as cropping and highlighting of audiovisual attention areas [25]. The aim of this class of adaptations is to improve the users’ ability to grasp the most important information. For example, when a large photograph is viewed on a small mobile display, it is difficult to recognise people posing in the picture. However, when each person is zoomed in across the entire display, it is easier to recognise them [26]. Cropping operation can be used for achieving this. The last group of signal processing mechanisms transforms the input signal at structural level, such as summarising and mosaicing a lengthy video stream [27, 28]. These operations attempt to provide a concise representation of large scale content in the forms of still images, shorter video segments, graphical representations, or textual descriptors that are more appropriate for a busy social media consumer. Moreover, this group of processing mechanisms also provides an easy means for deciding whether to consume the lengthy version of the social media content.

Besides the abovementioned classification, the adaptation mechanisms can also be categorised in terms of whether they are executed offline or online, and also depending on the purpose of adaptation. Offline adaptation is performed well before the content is consumed, often when the media is created. The objective is to keep as many adapted versions as necessary, so that users are given the option to choose the most adequate version for their preferences and needs. This approach is appropriate when online adaption is not feasible. However, the storage requirement increases with the proliferation of potential usage scenarios, each of which may have unique requirements. In contrast, the online adaptation is performed when social media is requested for sharing or consuming. The adaptation may be performed on real-time or non-real-time basis depending on the application. For instance, for streaming, real-time adaptation is necessary to enable pause-free continuous playback of the media. The disadvantage of online adaptation is the massive demand for computational resources for performing complex operations that need to be accomplished within a very limited time interval to guarantee the uninterrupted media playback. However, for on-demand applications that operate on the basis that users wait until the adapted media is ready before consuming it, the processing time delay pressure is less intensive.

Adaptation can also be classified in terms of the purpose of adaptation, such as content selection, content processing, and presentation adaptations are a few purpose-driven adaptation classes. Some of the adaptations are performed to assist users to select certain content. Summarising and mosaicing are classic examples. In this way, users can be given more informative ways of choosing the right content that they want to share or consume. Content processing may also trigger adaptations. For instance, a spatial resolution of a video needs to be reduced to add a decorative border around it. Similarly, creating a picture-in-picture effect not only requires spatial subsampling of one of the two videos (i.e. the secondary video) but also needs temporal resolution matching of the main and secondary videos if they are of different frame rates [29]. Adaptation for media presentation is required to match media characteristics to presentation devices or medium. For example, the layouts of web pages are often modified to better suit small displays such as those fitted on smart phones. Similarly, 5.1 multichannel audio needs to be re-rendered to match its features to a stereo sound reproduction system.

In light of the above discussion, the following subsections briefly describe the most frequently used signal processing mechanisms for realising personalised social media access.

3.1 Text-to-Speech and Speech-to-Text Transmoding

Traditionally, text-to-speech and speech-to-text transmoding have been used as human-machine interaction (HCI) tools for people with audiovisual impairments. Even though this tool was a life saver for those who were unable to read or hear, until recently they were not welcomed by the general public due to poor quality of experience. The robotic-sounding synthesised speech generated by text-to-speech transmoding tools were dull and often hard to understand. Similarly, speech-to-text conversion results were very unreliable in most practical usage environments. However, this attitude has changed in time due to rapid improvements in relevant algorithms. At the same time, powerful hardware platforms that can offer sufficient computational resources for these sophisticated algorithms to operate effortlessly are becoming ubiquitous. Therefore, deploying such algorithms on mobile devices, on which these technologies are most frequently used, is commercially viable nowadays. As a result, both text-to-speech and speech-to-text tools are becoming ‘must have’ tools for the social media users on the go. For example, an application that reads out incoming text messages is a standard tool in recent popular smart phones. Likewise, there is an increasing trend to listen to, rather than reading, the text posted on social networking sites. At the same time, those who are not fond of fiddly touch controls and keyboards on smart phones will soon be able to enjoy all-voice-based interactivity.

Even though both speech-to-text and text-to-speech transmoding are straightforward tasks for a human being, for machines, they are unimaginably hard computational operations that involve a number of sophisticated signal processing techniques. The text-to-speech transmoding is generally achieved using a synthesiser that mimics the human vocal system [30]. A simplified block diagram of a typical text-to-speech synthesiser is shown in Fig. 5. The input text stream is analysed by the pronunciation system, which produces a series of phonetic codes. These phonetic codes consist of phoneme codes and prosody indicia. Speech synthesiser produces the digital version of the speech signal using these codes. Finally, the digital to analogue converter converts the digital speech signal to its analogue form, which can be played by the speaker.

Fig. 5

Text-to-speech synthesiser

In contrast, speech recognition maps a continuous speech signal into a sequence of recognised words [31]. Modern speech recognition systems often use hidden Markov models (HMM) and the Viterbi algorithm to recognise words from speech signals. A simplified block diagram of an automatic speech recognition system is shown in Fig. 6. The feature extraction module, first, preprocesses the input audio signal. It performs operations such as noise cancellation, normalisation, and pitch correction. Subsequently, it extracts feature characteristics of the speech signal. The resulting stream of feature vectors is used by the search module. The search module performs the core acoustic pattern matching operations of speech recognition. Three types of models are employed by the search module to decode the speech signals as shown in the figure [32, 33]. The first type, the acoustic models, assigns probabilities to speech sound (phone). The second set of models, the lexicon models, specifies phone sequences for words, and the last type is the language models, which specifies the probability of a sequence of words for a given language. The post-processing module, subsequently, performs application-specific tasks including normalisation of output formats such as formatting date in de facto standard styles (e.g. ‘second of January twenty twelve’ into ‘January 02, 2012’).

Fig. 6

Automatic speech recognition system

3.2 Language Translation

Language often limits the audience for social media. Not everybody can enjoy a video clip with a soundtrack of an unfamiliar language. Therefore, automatic language translation can enhance not only the reach of social media but also the interactivity between different ethnic groups beyond linguistic boundaries. This tool can be used in many ways in social media domain, as shown in Fig. 7. The most straightforward adaptation is to translate text-based material into a text in a different language (i.e. the target language). Together with speech recognition and speech synthesis, more advanced uses are also possible. Automatic insertion of subtitles in target language is one of the examples of combined use of automatic language translation and speech recognition. Addition of speech synthesis to the abovementioned example also enables re-dubbing of video in target language.

Fig. 7

Use of language translation in social media

Automatic language translation is technically identified as machine translation, which is defined as the use of software to translate text or speech from one natural language to another [34]. The key challenge in machine translation is to produce output text or speech maintaining the semantic, pragmatic, structural, lexical, and spatial in variances [35]. Maintaining the meaning of the source text in the texting target language is known as semantic invariance. Pragmatic invariance refers to preserving the implicit intent of the text or speech. This includes conveying the politeness, intent, and urgency of the message. Maintaining the syntactic structure of the source text is identified as structural invariance. Lexical invariance defines the ability of preserving one-to-one mapping of words or phrases from source to target language. Preserving external characteristics of the source material, such as its length, location on the page, and (in case of speech translation) synchronicity with the associated video, in the target language, is known as spatial invariance. Success of maintaining the abovementioned characteristics governs the quality of translation. The quality of translation has not been high enough to gain complete satisfaction from the users yet [36]. A tremendous effort is still needed from the research community to perfect the technology for widespread use in social media applications. Nevertheless, machine translation tools are commonly available for the professional and non-professional use. Amongst them, Google and Microsoft translation tools, which are integrated into a number of online as well as offline service and tools, are perhaps the most commonly used translation tools. However, they are mostly text-to-text translators. Meanwhile, dedicated portable speech-to-speech translators, such as ECTACO Travel SpeechGuard TL-4, are also available in the consumer market. Smart phone applications such as SpeechTrans, which supports most European and Far Eastern languages, are also becoming available in the consumer market.

A number of machine translation technologies have been discussed in literature [37]. The rule-based approach defines a set of rules to map original text into the target. In contrast, statistical translation technology uses statistics based on bilingual text corpora. Another approach is the example-based approach, which uses bilingual corpus as its main knowledge base, and translation is achieved by analogy. Hence, it can also be considered as a case-based reasoning approach.

3.3 Transcoding

Transcoding can broadly be defined as an operation that converts one encoded signal to another. This is a vital signal processing operation to ensure any media content is available to its target audience, who may not be able to consume the media in its original form due to various reasons (usage environment constraints), including unavailability of the correct software, network bandwidth limitations, and processing power constraints. Transcoding can thus be used to address a number of the usage environment constraints. Hence, this operation has a number of target objectives such as format conversion, bit rate reduction (transrating), variable-bit-rate to constant-bit-rate conversion, spatial resolution reduction (resizing), and temporal resolution reduction. A classic use case for format transcoding has emerged from the growing number of smart phones that do not support popular Adobe Flash format. Users of the phones, which are shipped without any Flash support, are thus unable to view content in social media portals such as YouTube, unless the media are transcoded to a format that those phones support, such as H.264/AVC and MPEG-4.

Transcoding is commonly achieved in two stages: decoding the source media into an intermediate format and re-encoding into the target format [23]. Extra stage may be necessary in some applications including image and video resizing and cropping, audio re-sampling, etc. Most straightforward transcoding approach would be to fully decode and re-encode. This is the most feasible solution for achieving objectives like arbitrary resizing of video and transcoding between encoding standards with no common syntax. However, it is usually very costly due to the computationally intensive re-encoding operation. Thus, the computational complexity reduction in transcoding is one of the hottest research areas in transcoding research [23, 38]. Approaches such as reusing as much information as possible from the source content have been discussed in literature to address this issue [25]. On the other hand, transcoding objectives such as transrating while maintaining the same encoding format can be achieved without fully decoding (i.e. through partial decoding) the content [39]. Thus, two transcoding approaches can be identified depending on the intermediate format, namely, full decoding- and partial decoding-based approaches. These approaches are summarised in Fig. 8.

Fig. 8

Transcoding approaches

Some of the state-of-the-art encoding formats have been developed with support for low-complexity transcoding. A number of transcoding operations can be achieved simply by dropping unnecessary information in scalability extension of H.264/AVC [40] compatible video. These operations include common transcoding operations such as spatial and temporal resolution reduction and bit rate reduction. This is achieved utilising incremental encoding and smart packetisation approaches.

In general, transcoding is a lossy process unless the target encoding format is a lossless one. Moreover, if the transcoding is performed on-demand basis, computational complexity becomes a critical factor to consider before commercially deploying of such a technology. Considering, the above two factors, a measure to assess the efficiency of the transcoding operation, ${{E}_T}$, can be defined as follows:

$$ {{E}_T} = \alpha \cdot {{Q}_D} + \beta \cdot C $$

(1)

where ${{Q}_D}$ is the quality degradation, $C$ is the complexity, and $\alpha$ and $\beta$ are normalisation constants that assign the relative importance of the quality degradation and complexity, respectively. Based on the above definition, the lower the ${{E}_T}$, the better the transcoding technique. It should also be noted that quality degradation is linked to the complexity. For example, it is possible to reduce the transrating complexity of H.264/AVC-coded media by reusing motion vectors from the source media. However, this may incur a significant rate-distortion performance loss. The quality degradation can be minimised, however, by refining the motion vectors using rate-distortion optimisation, which in turn increases the computational complexity [41]. The state-of-the-art transcoding technologies make sure the loss of quality is marginal compared to the complexity reduction.

Even though transcoding is mostly associated with audiovisual media, it can also be easily applied to text-based media. For instance, a Microsoft Word document can easily be ‘transcoded’ to a HTML format that can be viewed on any device equipped with a web browser.

4 Adaptation Decision Taking for Heterogeneous Media Access

Adaptation decision taking is the brain of any context-aware content adaptation system. The role of an adaptation decision taking is multifold:

• 1. To select the most efficient and effective signal processing mechanisms from a large number of available mechanisms that effectively address the constraints described by the contextual information received from various context sensors

• 2. To determine the configuration parameter settings for the selected signal processing mechanisms (such as the output bit rate and encoding format for video transcoding)

• 3. To determine the most feasible location (i.e. at the server, a network node/proxy, or user terminal) for performing the adaptation operation

The ultimate objective of social media adaptation is to maximise the user experience under constraints described by the context. For instance, assume that a mobile user moves to an area where the data rate is not sufficient, due to poor signal strength, for the video stream he/she is consuming. This scenario can be addressed by reducing the bit rate of the content. From the signal processing point of view, though, there are a number of different ways to achieve lower bit rates including transrating and reducing spatial and/or temporal resolutions. Deciding the operation that produces the best quality video stream is the challenging task for the adaptation decision-taking stage.

To achieve the abovementioned objectives, first of all context has to be interpreted to uncover constraints, which calls for media adaptation. Technically, this is identified as context reasoning. Context reasoning is defined as inferring new information relevant to the application from a vast amount of data received from various context sensors [42]. Data received from various context sensors is often either incomplete or inaccurate. Thus, the first task of context reasoning is to perform corrective measures to overcome potential issues to improve the validity of the context. Subsequently, raw data has to be transformed into meaningful information. For the adaptation decision-taking application, constraints imposed by the detected context have to be inferred. Literature discusses a number of techniques that can be used for context reasoning such as ontology-, rule-, and case-based reasoning [43].

Once the essential contextual information is deduced, the next stage of adaptation decision taking is to determine the output format and associated configuration settings (e.g. for video: encoding format, encapsulation/container format, bit rate, frame rate, spatial resolution). In some cases, finding a suitable signal processing technology for performing certain types of adaptations in a single step may not be possible. In such scenarios, it is necessary to apply a number of signal processing operations on the source media in a sequence to generate the target format. As a result, the adaptation decision taking should consider the available set of technologies and their limitations, as additional set of constraints. Hence, the following extra information must also be derived during adaptation decision taking:

• Sequence of signal processing operations that are required to perform in order to generate the required output adapted bit stream

• Sequential order, in which the signal processing operations are applied

Adaptation decision taking can be implemented as an integral part of the signal processing mechanism, as shown Fig. 9. This integrated entity is called the media adaptation unit (MAU) [44]. For example, a mobile service provider can commission transcoders at base stations to dynamically control the spatial and temporal resolution, and the bit rate of the media before forwarding them to mobile phones to better utilise the limited base station capacity. Adaptation decisions are taken locally, and therefore sporadic context changes can be promptly served by dynamically changing the configuration settings.

Fig. 9

Adaptation decision taking and signal processing as an integrated unit

An alternative to the aforementioned integrated architecture is the decoupled architecture, in which adaptation decision taking is implemented as a stand-alone service, often by a dedicated module known as adaptation decision engine (ADE) or adaptation decision-taking engine (ADTE) [45]. The advantage of this architecture is that the signal processing elements (often identified as adaptation engines – AEs) scattered throughout the communication chain can be coordinated to optimise the overall performance of the network. This can be explained with a scenario, where there is a bandwidth limitation in a downstream edge (e.g. user’s access link). This scenario can be addressed by transrating media content at the network edge. However, the media servers still need to deliver high-bandwidth media all the way to the edge network. This is a waste of precious communication resources in the core network as well as the social media service provider, given the fact that the high bit rate version cannot be delivered to the user as it is. Alternatively, the adaptation can be performed at the server-side for improving network resource utilisation. Now, assume that more than one user of the aforementioned edge network consumes the same media, and a number of them do not have any bandwidth issues. In such a scenario, it is more appropriate to perform the adaptation back in the edge network. Therefore, the place of adaptation is context dependent and is also an important decision that adaptation decision taking should consider. A stand-alone adaptation decision-taking architecture can provide necessary flexibility to achieve this task. Figure 10 illustrates this architecture in detail while also showing the most appropriate location of adaptation.

Fig. 10

Adaptation decision taking as a stand-alone service

Nevertheless, the advantages of both integrated and stand-alone adaptation decision architectures can be obtained by commissioning a two-tier distributed adaptation decision architecture, in which a set of MAUs are overseen by a top-level adaptation decision-taking service. High-level decisions such as choosing the signal processing mechanism (e.g. transcoding) and their locations can be top-level decisions. Low-level parameters such as the output data rate can be determined in the lower layer by the MAU.

Out of the abovementioned architectures, the most feasible choice of social media adaptation decision taking is the integrated architecture since necessary infrastructure support is not yet available to realise the stand-alone architecture. The adaptation is currently possible at the content adaptation modules available at social media servers, edge networks, and user terminals. Coordinating those adaptation modules is also virtually impossible currently. Unavailability of suitable protocols to communicate amongst the adaptation decision-taking servers and signal processing modules is deemed to be the major problem at the moment. Besides, security implications of intercepting content have to be addressed in order to gain mainstream recognition for both stand-alone and distributed architectures [46].

4.1 Adaptation Decision-Taking Approaches

The adaptation decision-taking problem is a constrained optimisation problem described as follows:

Given any input media, define the output media that intended users can consume and enjoy under the constraints defined by the contextual information in such a way that their experience is maximised.

According to the above definition, the prime objective of adaptation is to maximise the user experience. Two different approaches have been suggested in literature:

• 1. Knowledge-based adaptation decision taking

• 2. Utility-based adaptation decision taking

The former technique uses a set of predefined rules [6] to find a suitable set of adaptation parameters to satisfy the constraints, while the latter attempts to define the bit stream format that maximises the utility [47, 48].

The knowledge-based approach considers adaptation decision taking as determining adequate signal processing sequences as a classical state-space planning problem [49]. A sequence of operations is determined to transform any given media to a format that maximises the user experience. Estimating user experience is a non-trivial problem by itself. Considering the practical difficulties, simple approximation models are often used for quantifying the user experience even though they do not accurately predict every individual’s perception. Some examples are peak signal-to-noise ratio (PSNR), peak signal-to-perceptible noise ratio (PSPNR), and video quality metric (VQM) for video and signal-to-noise ratio (SNR) for audio [50].

The utility-based adaptation defines three important spaces – adaptation, resources, and utility, as shown in Fig. 11 [47, 48]. The adaptation space defines all the possible signal processing options that can be performed on any given media. Spatial and temporal resolution reduction and transrating are a few adaptation examples (a1, a2, and a3 axis, respectively, as in Fig. 11) for encoded video. Each point in adaptation space has corresponding points in resource and utility spaces. The resource space represents the resource requirements, including communication bandwidth requirement and minimum specifications of the user terminal(e.g. processing power and display resolution). Utility is the user satisfaction after the adaptation. It can be measured either subjectively or objectively. However, subjective method is not practical for serving for social media adaptation. Therefore, using objective measures, such as PSNR, PSPNR, and VQM, is the most feasible alternative. The utility function maps resource requirements into utility for each of the adaptation operations. The principle of the utility-based adaptation decision taking is simply to find the adaptation operation that produces the best utility while not exceeding the resource budget.

Fig. 11

Adaptation-resource-utility space concept and the utility function

Even though the adaptation decision taking is classified as knowledge-based and utility-based, in a broader sense, the utility could well be a part of a knowledge-based approach [51].

5 Challenges in Social Media Adaptation

Subsequent to identifying and introducing context-aware content adaptation as one of the key technologies that enables personalised access to social media in the previous sections, it is now appropriate to examine a suite of prime issues that may set significant challenges for performing social media adaptation in the broader sense. As clearly stated earlier, social media is strictly a user centric concept, and thus it bears wide-scale one-to-many or many-to-many user-to-user communication and media sharing properties. This leads to the need for developing personalisation techniques to cater for an immense variety of user preferences, needs, characteristics, profiles, etc. Providing one solution to fit all of those is not possible due to both technological and non-technological (in other words, social) reasons, which constitute the prime challenges against social media adaptation, unless they are clearly identified and addressed.

5.1 Technological Challenges

Technological challenges are the challenges that may affect or inhibit the operation of the social media adaptation tools due to technological reasons. There may be several reasons causing such challenges; however, the following four have been identified in this subsection as the major issues to highlight.

Digital rights management (DRM) provides a set of access control technologies, which are used by the content providers to control the use of their digital content by unintended and unauthorised users in a digital value chain [52–56]. From this perspective, DRM supports intellectual property protection, which allows only authorised users’ access and use of the protected digital content based on the rights expressions available, which govern the DRM-protected digital content [57]. Although intellectual property protection is important and necessary, particularly in today’s social media world where even ordinary consumers have become content producers, it also has an inhibiting effect on the widespread access, sharing, and use of social media by users, sometimes even within the same social circle as the content provider. Thus, DRM can be a major limiting factor for a good personalisation practice on the social media content during adaptation, which requires particular attention. In literature, the importance of this challenge has been noted through specific research publications that propose methods to perform DRM-enabled media adaptation [45, 46, 58]. In general, those methods look at innovative ways to adapt media content while respecting the authorisation requirements for some types of adaptation operations, which are imposed by the DRM regulations that are set by the content providers.

Secure media applications via encryption of the media content are another limiting factor for the widespread access, sharing, and use of social media. Encryption also influences the required content adaptation operations, so as to provide personalised social media access to wider ranges of users. An encrypted content cannot be decrypted without sharing the encryption key with the user terminal. Without having such a key, it is very difficult to adapt the content if the media properties and media streams cannot be accessed by the adaptation middleware or tool. In literature, several research publications have been provided to address this significant issue, which investigate methods for adapting secure (i.e. encrypted) media content without the need for decryption or through partial decryption [59–63]. Nevertheless, true personalisation demanded by the social media users needs access to the media content itself, so that necessary adaptation can take place as requested. It is possible to achieve this by sharing the adaptation load with the content encoding and encryption stages at the content provider’s side to some extent. Alternatively, the encryption key can be shared with the third-party adaptation middleware, which in turn puts sensitive content at risk of potential attacks by those who should not have the access. For a more generic solution to the problem, a fully automated blind secure media adaptation solution is yet to be realised, so that both proxy/gateway-based and user terminal supported personalised adaptations can be performed effortlessly and effectively.

Copyright issues also pose notable challenges for adapting social media content. As with DRM-based intellectual property protection, copyright protection of digital content limits the possible media adaptation operations that can be performed on social media that is shared amongst online social communities. Legitimate adaptation operations can still be applicable, as the copyright terms permit; however, it cannot be guaranteed that such operations will still have a wide enough scope to be able to address all of the diverse preferences or requests of a wide range of users.

Last but not least, new challenges arise for indexing, search, and retrieval of tremendous amounts of social media content uploaded everyday by many users for sharing with diverse online communities. Social media content adaptation technologies developed to cater for personalised access to social media hence should also operate hand-in-hand with personalised and fast social media search and content retrieval solutions, which are further elaborated in the other chapters of this book.

5.2 Non-technological (i.e. Social) Challenges

Non-technical challenges are those related to social aspects that may inhibit the operation of content adaptation tools for achieving efficient personalised access to social media. Below, a representative set of the most prominent challenges has been examined in detail to highlight the types of issues that can be often encountered during social media adaptation. However, it should be noted that certainly this does not provide a comprehensive list, which can be extended by the readers and upon their experience gained through various user activities in social media access, sharing, and distribution platforms.

Under no circumstance, the privacy of users should be jeopardised during social media adaptation for personalisation or any other purposes. As described in the earlier parts of this chapter, context-aware content adaptation relies on an adaptation decision-taking mechanism as an integral part of the overall adaptation process, which takes inputs from various external context providers for deciding on the best suitable adaptation method for a given situation. In addition to terminal capabilities, network characteristics, media content specifics, and natural environment properties, the diverse range of contextual information also comprises sensitive data on users’ personal profiles, and thus their special needs, preferences, and personalisation requests to assist with adaptation. The contextual information is usually stored at a suitable location (e.g. at a dedicated middleware, a third-party context provider’s database, partially on user terminal) in the social media access, sharing, and delivery chain, and hence obtained and processed when required by the adaptation decision mechanisms.

It is very important to collect and store such personal contextual data with necessary protection due to today’s growing concern on user privacy, where exchange of sensitive personal information amongst different systems has become a common practice, particularly in heterogeneous social networking scenarios [64, 65]. Contextual information requires similar treatment in terms of protection and privacy issues, and there are examples of different generic privacy protection system implementations reported in literature [66–69]. The good practice for protecting privacy is to define a privacy model, which identifies the information to be protected, so as to set strict privacy rules for protecting certain context depending on who it is from, who it is addressed to, and what its intended use is, while also respecting the necessary issues related to the ethics.

In social media access, sharing, and distribution scenarios such as social networks, the common trend is that user is expected to be responsible for defining his/her own privacy preferences. However, this may not always be practical to implement, as can be noted from the growing concern amongst social media and social networking users today. Instead, such sensitive information should be defined beforehand, and necessary privacy protection systems should be put in place and made transparent to the users while performing social media content adaptation. Here, the aim should be to protect the privacy of social media users while adapting their content for providing personalised access by protecting and not revealing their personal contextual data.

Privacy protection can thus pose significant challenges for supporting personalised access to social media through context-aware content adaptation. Large numbers of social media users and their very diverse personalisation requests due to various personal preferences may equally set strict challenges for adapting the content as required. To further elaborate on this with an example, one can assume a scenario where some media content is uploaded for sharing through social networking. It is likely that some of the users in this online social network would prefer to receive a part or parts of the shared content that concerns them while the others would like to receive the other parts or all of it. Within those groups of users, some prefer to receive only the audio content due to their prevalent contexts (e.g. mobile, driving or busy doing something else hence cannot watch, eyesight problems, disabilities), and some others are happy with the visual content at the expense of text or audio components due to other yet similar reasons. Indeed, this example can be taken a few steps further down to address each user’s personalisation request individually, who accesses the one piece of content shared in such a heterogeneous online social community. In this way, the complex dynamics and multidimensional nature of media content adaptation becomes much clearer, where while responding to each and every personalisation request is very important, satisfying both the content provider’s set rules and other users’ comfort and their personalisation aspects is equally needed. A similar example can be formulated around another social media content that is shared, which contains scenes of an obscene nature for a particular group of users while it is acceptable to others. Thus, it is the sole responsibility of the adaptation decision mechanism to make use of the personal profiles provided in the form of contextual data accordingly, so as to decide on the most adequate adaptation solution to respond to every type of personalisation request in the most effective way possible. Evidently, the greater the diversity of requests for such personalised adaptations, the harder the challenge becomes that is faced by the social media content adaptation technologies.

Inaccurate adaptation of social media may result in adapted content that is completely ‘out of context’, which in turn may be disturbing for some individuals with potential psychological or social impacts on their online social behaviour. This may particularly happen if content adaptation is performed where semantics of the content are not carefully managed due to lack of availability of necessary contextual data inputs for inferring high-level context from low-level contextual information. It is of paramount importance that the content adaptation solutions that are selected and applied should provide acceptable outcomes to the precise needs of the users and their personalised social media access requests. For instance, a chosen adaptation operation leading to cropped images, video material, or random extracts from audio data, as requested by a mobile user with a small resource-restricted device, may end up with changing the entire meaning of the content compared to its complete version. Similarly, another adaptation operation performed to provide a selection of a view or a subset of views from multi-view/free-viewpoint video content without considering the overall message conveyed in the shared media may result in providing the user with the wrong angle to observe some events in the delivered visual scene, which may then turn out to be open for misinterpretations. This may cause unintentional stir in user’s expectations as well as his/her mood and attitude towards the content that is shared through social media access technologies. To avoid this from occurring, social media content adaptation systems should also pay utmost attention to the semantics of the content that is being adapted, so as to retain its meaning as a whole, which may not always be an easy task to achieve, particularly under stringent low operational delay and computational complexity requirements.

6 Conclusion

In this chapter, context-aware content adaptation has been presented as an enabling technology for providing personalised access to social media. For this purpose, the rationale behind the need for adapting content and possible middleware structures that perform such operation in social media access, sharing, and delivery scenarios have been introduced firstly. Following, the importance of the use of contextual information and the different context types that are used in performing content adaptation in such scenarios have been emphasised. Next, discussions have been focused on describing the techniques for actual social media adaptation operations in detail, where the content adaptation tools used and the decision-taking mechanisms for selecting the most suitable adaptation operations for a given situation have been revealed. Last, both technical and non-technical (i.e. social) challenges have been highlighted, which are thought to influence the accuracy and performance of those possible social media content adaptation options. Particularly, in this final section, a number of key pointers are provided to indicate open issues for future research, so as to address those highlighted challenges for realising true personalised social media access environments. It is worthwhile mentioning here that due to the multidimensional and multidisciplinary nature of the open issues, providing personalised access to available social media through context-aware content adaptation calls for inter-disciplinary and multinational/multicultural research and technology development efforts for finding effective solutions to the diverse technological and social challenges identified in this chapter.