Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

4.1 Introduction

This paper addresses the issues on developing web-based collaborative content authoring in multi-device environment and utilizing metadata provided in uploaded media, as well as providing social contents recommendation using metadata provided in the user’s Facebook account. Our proposed system is considering a distributed user interfaces (DUIs) [1] for collaborative authoring, which is based on the concept of UI component adequate for the physical device characteristics and social media recommendation scheme from SNS such as Facebook.

This paper describes our approach for web-based social collaborative authoring technology and shows some current research results.

Consider some memorable events such as wedding ceremony, high school graduation or academic fair that involves a group of friends who took photos at the event. Each friend took a photo based on their own perspective and their own point of interest. Each friend tends to have different interest, so photographs taken by different friends will likely cover the event from different perspectives. Hence, collecting the photos from various sources is needed to comprehend the whole event from various perspectives. The resulting photos also tend to be distributed in each photographer’s personal drive. It is cumbersome to obtain their photos one by one. And then, to obtain friends’ multimedia, each user uses own device. At this point, each user may use various kinds of devices. Some of the users use desktop in their home and office. However, some of the users use mobile devices for publishing their multimedia and obtaining their friends’ multimedia from SNS.

Fortunately, the widespread usage of SNS helps photo sharing among friends. Using the photo content uploaded in the SNS, the users can collaboratively combine the photos to create a video content that has personal meaning. To create narrative video using photos on a certain event, the authors need related photo content about certain topic/event to support content authoring. However, to our best knowledge, no current authoring tools support recommending media content from SNS, such as Facebook. An SNS-based content recommendation system for authoring is needed in our collaborative authoring system.

The goal for developing recommendation system is to help the collaborating authors by providing related photos from Facebook. The recommendation module is a novel method for video authoring. The recommendation module suggests related photos from SNS based on the keyword in the analyzed Collaborative Authoring Metadata (CAM) [2].

Kaplan and Heinlein [3] categorized social media into various types, including Social Networking Services (SNSs). The content in SNS has deeper social meaning than content-communities social media, because it has higher self-presentation and self-disclosure. One of the most popular SNS is Facebook. Statistics presented by Hachman [4] claims that Facebook has 901 million users. Parr [5] reported that 250 million photos are uploaded every day on Facebook. The photo uploaded in SNS (e.g. Facebook) tends to be much more personal and have deeper social relationship meaning compared to content community social media (e.g. Flickr). For this reason, in view of social collaborative authoring, Facebook’s photo contents are prominent resources for the content being authored due to the amount of contents it contains and the social relationship meaning of the contents to the users. The next challenge is how to recommend related photo contents to the authoring system.

Mobile devices are currently widely used. In a January 2012 statistics provided by Ansonalex.com, there are 5 billion mobile phones used worldwide, and 1 billion of them are smartphone. Therefore, the usage of mobile devices to support daily activities is likely increasing, including the usage for collaborative purpose.

As DUI application, this paper describes the development of Facebook photo recommendation for collaborative social video User Created Content (UCC) authoring tool. Several things are done to achieve this goal, such as (a) Studying the behavior of Facebook users in sharing photo content to their Facebook account, and (b) Designing and implementing recommendation mechanism for getting co-event content from Facebook and prioritizing the result.

This paper also describes collaborative method between mobile users and desktop users. Mobile users can be recommended multimedia from SNS and participate collaborative authoring via web environment. Current mobile devices have a rich set of features, such as GPS, camera, microphone, wireless networks (Bluetooth, Wifi, 3G, LTE) with decent computational resources. In view of collaboration, mobile device advantages can be used to support collaboration. The users can support content creation by doing one of the authoring tasks: video authoring, audio authoring, and image authoring. The users can support content authoring by providing various multimodal contents, such as video, audio, image and even text. In our system, user can participate in collaborative authoring task with their friends which use various kinds of devices.

4.2 Related Work

There are many researches on collaborative authoring [6–13] and collaborative softwares [14] that support various purposes. Among them, the typical web-based document collaboration tools are Google Docs and Wiki. The Google Docs provides simultaneous document editing; however there is lack of communication to share the editing intention. The Wiki has a lack of contents sharing during authoring process and also lack of group management between authors.

In 2011, the Creaza VideoCloud Platform is introduced [15], which is a tool for collaborative video authoring on the web. Lately, this tool is called as WeVideo [16] as a commercial solution. The main feature of WeVideo includes web-based collaboration, video authoring, and utilization of cloud. However, WeVideo is lack of communication to share collaborating the editing intention and comments among collaborative authors.

Stupeflix [17] is a web application to make videos in a few clicks. This solution imports directly from Facebook, Flickr, Picasa or Dropbox. User can add text, maps, voice-over, images and videos. This one also provides customized preview and free videos for download in HD. Stupeflix provides open API for developers. This solution provides open APIs for developers. This solution does not support collaborative authoring; however, it supports the coordination with SNS (social network services) contents for video authoring.

4.3 Collaborative Contents Creation Using Web-Based Distributed User Interfaces

Our general direction can be seen in Fig. 4.1. The users have multiple devices (e.g. tablets, smartphones, PCs and notebooks) with different display size, computational resource, and features. Every devices connected to the internet, and the internet connects the users to several services, such as mobile messaging service, collaborative content authoring service, and social networking service. The users can create a content using web based collaborative authoring service anywhere, using any devices that connected through the internet. Since the user might not feel convenient using the UI developed for desktop in their mobile devices, component based specific UI for mobile devices are developed.

Fig. 4.1
figure 1

General direction of the proposed system

Full size image

In view of DUIs, for heterogeneous device/platform, a concept of UI component is used and its component can be downloaded to devices according to the authoring purpose and device’s physical characteristics. In other words, functionalities of collaborative authoring can be divided into component. For example, the authoring of multimedia contents handles several media, such as image, video, audio and text. In the desktop environment, the authoring tool provides all the functionalities for multimedia in one application UI. However, in case of mobile devices, it is not possible to provide all multimedia authoring functionalities in mobile device with small screen and low computational capability.

Another consideration is the authoring system did not have the capability of adapting the UI according to specific editing part for the user. Some authors might be expert to provide audio enhancements on the project (audio authoring), while the other authors are excellent in narrative visual storytelling (video authoring), and the other users might know many things that could be used to provide textual information on the project (textual authoring). In this case, it is needed to provide adaptability of the interface based on the users’ intention (or expertise). For supporting the expertise in collaboration, our system supports three interfaces, Audio Authoring User Interface, Video Authoring User Interface, and Textual Authoring User Interface.

Collaborative work needs to share knowledge, experience and abilities to achieve common goals among users. It is important to share user’s characteristics for collaborative authoring on distributed environment among users. For collaborative authoring, our system designed CAM (Collaborative Authoring Metadata) that includes authoring intention, name of author, created date, time, location, mood, with whom and so on. Each of users can upload and create their own contents (Video, Image, Audio and Text) to collaborative authoring space. When user upload and create their own contents, CAM is created as additional knowledge and experience.

It should be noted that although there are personally meaningful multimedia data in our social networking sites; the current authoring tools are incapable of recommending multimedia contents from our social networking sites, such as Facebook. This paper addresses the issue of the related contents recommendation from social media services during the collaborative authoring. The above mentioned CAM is used for the recommendation of social media contents.

4.4 Recommendation Technique Review

For the contents authoring, the recommendation of appropriate related contents are needed. Recommender system is a software tool and technique that suggests items to be used by a user [18–20]. The term “Item” refers to what the system recommends to users. In most cases, a recommendation system only focuses on a specific type of item (e.g., movies, news or music). In the past few years, recommendation system has become a valuable means to cope with the problem of information overload [21].

The interest towards recommender systems has been dramatically increased lately, as indicated by some facts. First, recommender systems play an important role in such highly rated internet sites (e.g. IMDb, Amazon.com). Second, there are dedicated conferences and workshops related to the recommendation system field (e.g. ACM Recommender Systems – RecSys). Third, college courses that dedicated entirely to recommendation system are offered at higher education institutions around the world. Lastly, there have been several special issues in academic journals that cover research and developments about recommendation system [21].

Recommendation systems have several differences with search engines. The goal of search engine is to answer user’s ad hoc queries, while recommender systems are created to recommend services or items to user. The input of a search engine is defined as a query, while recommendation systems also rely on user preferences that defined as a profile. Output of a search engine is ranked items relevant to user’s need, meanwhile, in recommendation systems, the items are ranked based on user’s preferences. Search engines rely mainly in information retrieval-based methods, while recommendation systems rely on several methods, such as information retrieval, machine learning, and user modeling [22].

There are two major approaches for recommendation systems. First, collaborative filtering based recommendation systems as described by Goldberg et al. [23], and Second, content-based filtering based recommendation systems as explained by Pazzani and Billsus [24]. Collaborative filtering uses data from another user with similar preferences (e.g. Amazon.com’s item recommendation). Collaborative filtering-based recommendation systems identify users whose preferences are similar to the current user and recommend items that have been liked by identified users [25]. Meanwhile, content-based filtering is based on the description of the item and a profile of user’s interest (e.g. Internet Movie Database movie recommendation). Content-based filtering-based recommendation system tries to recommend similar item to those a given user has liked in the past [25]. Some works use tags as content descriptors for collaborative filtering, such as work by Firan et al. [26] shows that tag-based profile is capable of producing better personal recommendations on Last.fm compared to conventional recommendations. Meanwhile, Guy et al. [27] use related people and related tags to recommend social media items (blogs, communities, wikis, bookmarks, files) using hybrid approach (both collaborative filtering and content-based filtering). After evaluating the result, they found that tag-based recommendation provides better item recommendation, and recommendation based on combination of people and tags provides slightly more interesting recommendation with less already-known items.

Lerman et al. [28] worked on recommendation system that tried to solve ambiguity caused by homonyms and polysemy in Flickr tags. Their work uses hybrid approach (combining collaborative filtering and content-based filtering) based in contacts and tags. Recommendation based on users’ contacts has proven to significantly improve the relevancy. In tag based part, a probabilistic topic model that predicts the users’ desired contexts is developed. The probabilistic topic model is based on previous tags used by the user and to which group the user assigns his/her photos into. The result for this is a model that interprets the keyword as intended by the user (not biased by either homonym or polysemy). Thus, the precision of recommended item increased. In this work, comment and favorites were not utilized and there was no way to handle uninformative tags (e.g. “Let’s Play”). Gursel and Sen [29] proposed another recommendation system which is also based on Flickr. They developed an agent that observes the user’s past activities and observes rating and comments provided by the user. As a result, photos are recommended in order, based on user preferences. Unfortunately, user with lack of past activities may have irrelevant agent. And also, the content source is derived from Flickr, therefore may not have a deep social meaning compared to SNS websites like Facebook.

4.5 Results

4.5.1 Our Social Collaborative Authoring System

This paper describes an architecture which can support the concept of DUI and links with SNS, such as Facebook. This architecture is provided in Fig. 4.2.

Fig. 4.2
figure 2

Architecture of the social collaborative authoring system

Full size image

The proposed system consists of web-based DUIs, web server and social database.

Web-based DUI provides a space to create project of collaborative authoring, publish the content, and manage authors’ accounts. In more detail, AUI (Authoring User Interface) is developed for desktop PC and mobile devices. Authors can store their resources (audio, photos, and videos), CAMs and friend’s information in the social DB. The web server links web based DUI and social DB, and includes the modules for collaborative authoring system.

Web-based DUI can be composed according to the user’s device. In case of desktop PC, user can use web browser in which all the authoring functionalities are provided. However, in case of mobile devices, user can select the DUI component according to the user intention. For example, the audio authoring user can only download the audio AUI and perform the collaborative authoring. Here, the pre-authored video and text content are provided as a reference in the timeline.

The web server consists of SCS (Social Collaborative System), MAS (Media Authoring System) and CMS (Contents Management System). The SCS includes collaborative project management module and group management module. These modules implement collaborative functions on the web. When a user searches for co-authors, group management module requests author’s information at the social DB and provides appropriate author information to the requesting user. The collaborative project management manages group of the project.

The MAS includes authoring module, recommendation module and CAM module. The authoring module provides editing capability and preview of edited content. The CAM module creates CAM, analyzes created CAM and displays this CAM information systematically for collaborative authoring. Using these CAMs, authors can exchange their authoring intention and information of each media. CAM is provided by authors during media (image, video or audio) upload. Our system defines and stores CAM using XML.

In case of creating narrative story using images, the authors need related images or videos about certain topic. Our recommendation system can help the authors by providing the appropriate image or video from social media services, such as Facebook. The recommendation module is a novel method for media authoring. The recommendation module searches related images from Facebook based on the keyword of the analyzed CAM. During the authoring process, each author can have recommendation with related images and sound from Facebook based on the CAM. For example, the author can be recommended with some Facebook photos that were taken by other participants, which include similar metadata.

CMS includes an account management module and a media management module. Our system is based on open source video editing tool (Moviemasher [30]) for implementing authoring module and Drupal [31] for implementing CMS.

Figure 4.3 shows UI of desktop PC. As shown in Fig. 4.3, our system supports CAM and recommendation of contents from Facebook.

Fig. 4.3
figure 3

Collaborative authoring tool for desktop PC

Full size image

4.5.2 Mobile UI

Our system supports collaborative authoring using smart phone like iPhone and Android phone using web browser. Figure 4.4 shows whole UI menus for collaborative authoring in the smart phone.

Fig. 4.4
figure 4

Web app for collaborative video authoring

Full size image

Due to the small screen size of smart phone, user can use authoring component based on the authoring media, like image or audio. According to the user’s authoring media type, user can select authoring UI, such as audio, or image and download it in his/her smartphone. Then, he/she can perform collaborative authoring only in its authoring media UI. Figure 4.5 shows image authoring UI and audio authoring UI.

Fig. 4.5
figure 5

Image and audio authoring user interface for smartphone. (a) Image AUI. (b) Audio AUI

Full size image

4.5.3 Invitation of Friends for Collaborative Authoring

For supporting collaborative authoring, our system supports friend or expert invitation in the authoring software. Figure 4.6 shows friend/expert invitation UI. Here, KakaoTalk, widely used message system, is used for sending invitation message and corresponding URL. When friend/expert received an invitation message, he/she can join the collaborative authoring simply by clicking the received message which links to an URL of web authoring space.

Fig. 4.6
figure 6

Expert friend invitation UI

Full size image

4.5.4 CAM and Facebook Photo Metadata

This paper also addresses the coordination of our collaborative authoring system and current Social Network Services such as Facebook, Flickr etc.

In Facebook, each user has many friends and shares several kinds of contents with one’s friends. So, for creating collaborative UCC, it would be also useful to use our friend’s Facebook album as a social database. For this, our system provides coordination of our collaborative authoring system and Facebook photo album.

Here, participants’ Facebook photos are accessed using Facebook API.

Our system supports the collaborative authoring based on the CAM. In Facebook album, each photo can have several metadata information such as time, location, likes, tagged person, comments and so on. So, these metadata of Facebook photo can be used as CAM for our collaborative authoring. Using these Facebook photo metadata, our system can search and collect the related photos of our friends from Facebook album and create social UCC using these searched photos.

Figure 4.7 shows an example of CAM created by users. According to the user’s situation and state of mind, the CAM can be created differently. For example, user1 creates upper CAM (a) and user2 creates lower CAM (b) in Fig. 4.7. As shown in the Fig. 4.7, user1 and user2 attended same event that is held at the same place. However, they have different feeling and spend event with different friends. Our system can use these different CAMs in collaborative work among distributed users. These CAM can be used appropriately for the collaborative contents authoring.

Fig. 4.7
figure 7

Examples of CAM created by two users

Full size image

Figure 4.8 shows a basic concept of recommendation system based on CAM. Our system includes Facebook contents recommendation engine using CAM. The detail of our recommendation engine will be described in another paper.

Fig. 4.8
figure 8

Facebook recommendation scheme with CAM

Full size image

4.6 Conclusions

This paper describes DUI issue for developing web-based user interface into collaborative social authoring. Our system provides web-based collaborative media editing environment and adopts CAM to communicate authoring intention and comments among collaborative authors, then coordinates with Facebook photo album. Our system addresses issues that arises in multi device authoring and proposes DUI for collaborative authoring, which has adaptability of the system to be used in multiple platforms and space.

Our system also introduces content recommendation scheme from Facebook during the collaborative authoring. The recommendation system for Facebook photos is developed by using several metadata available on Facebook. Content-based filtering and Collaborative Filtering is done sequentially to provide the recommendation. Instead of only using relevancy with the context, some social parameters like how close the relationship of the uploader to the user and how many interaction on a photo is measured to determine how interesting a photo is. Hence, it can provide relevant recommendation to be used as content resource for video authoring. After this work has done, web-based collaborative video authoring environment has developed and CAM has been adapted to match with social metadata available in Facebook. User can refer to CAM information to seek content recommendation from Facebook with a good accuracy from various perspective of the content to be authored, and based on this content; they can create content using relevant photo recommendation result.

Nowadays, the social curation technique is receiving much interest in view of collecting and reorganizing social contents in distributed and heterogeneous SNSes environment. Currently, we are now developing storytelling system using social curation technique. The future research issues include how to collect and group the SNS contents from distributed and heterogeneous SNS contents and how to provide collaborative storytelling system by using distributed multi-devices.