1 Introduction

Nowadays, with the enormous growth of Social Networking Sites (SNSs) such as Facebook and Google+, millions of users are sharing information with friends and followers creating an enormous stream of data in real-time. These data vary from personal news (such as status updates) to global news (such as news about politics, science, sports, technologies, etc.). If we consider the social data stream of a single user from her friends, only a fraction of it is relevant and interesting, while the rest of the stream results in social data overload to the user. Personalized filtering is a mechanism that aims at solving these challenges of social data overload by recommending the most relevant content to a user. It has been implemented by many social media sites, such as Facebook, Digg and YouTube.

A majority work on personalized filtering has been done on improving the accuracy of the filtering models through better modeling user preferences (Gauch et al. 2007). The underlying assumption is that providing enjoyable items correctly could lead to an increase in user’s satisfaction. In recent years, a growing trend in personalized filtering research is to consider factors, other than accuracy, that contribute to the quality of filtering, because an overt focus on accuracy has limitations: it fails to consider human desires for variety and leads users to only receive information that matches their pre-stated preferences but fail to be exposed to new topics (referred to as the “filter bubble” problem) (Pariser 2011; Zhang et al. 2012; Matt et al. 2014; Loepp et al. 2014). Moreover, current widespread personalized systems operate invisibly (Kay and Kummerfeld 2013). For personalized filtering, the platforms usually do not create awareness about what is being hidden from the user. They are black boxes and provide no explanation about what data is filtered and why. The importance of awareness and explanation has been discussed in previous studies (Scheel et al. 2014; Johnson and Johnson 1993), that lack of such transparency can lead to reduced user satisfaction (e.g., trust toward personalized filtering) in the system (Sinha and Swearingen 2002; Kay and Kummerfeld 2013; Lim et al. 2009). Therefore, it is necessary to allow users to take control of the filtering process and its outcome (Dooms 2013; Loepp et al. 2014).

In this paper, we aim to propose a visualization of filter bubble and answer the following three questions.

  1. (1)

    Can a visualization of the filter bubble be built as an effective technique to create user awareness, provide her with explanation and control of personalized filtering?

    The main purpose of personalized filtering is to reduce the social data overload by recommending only the relevant content. But showing what is hidden and filtered away from the stream can increase the social data overload problem. Therefore, the main challenge is to find an effective visualization technique that can be seamlessly integrated into the activity stream without contributing additionally to the social data overload. What is the right amount of detail to expose in the hidden filtered social data and its explanation? How do we organize these hidden filtered social data? What type of visualization is effective to display the hidden social data stream in an understandable way for the user? These issues can be explored through theoretical design and experiments with users.

  2. (2)

    Can a visualization of personalized filtering increase the user’s trust in the filtering system?

    There is the possibility that some of the hidden filtered social data are being wrongly classified as undesirable. We believe that showing the hidden filtered social data using an effective method, which does not cause the overload or undo the advantages of the filtering, will provide transparency of the personalized filtering to the user and increase the user acceptance of the system.

  3. (3)

    Can a visualization of personalized filtering alleviate the “filter bubble” problem?

    As the activity stream is personalized according to the user’s interests, the user will ultimately only see activities related to her interest and will have no opportunity of discovering new interests. This will lead to the “filter bubble” problem where the user is trapped in a world filled with only items matching her interests. By exposing (some of the) hidden filtered social data, the user will become aware of the model that the system has of her, and may consciously decide to explore items from other areas by changing interactively her model and it will open the avenue for discovering new interests. As a result, the user would be able to come out of her filter bubble.

This paper presents an interactive method to visualize the personalized filtering in Social Networking Sites to create awareness, explanation, and control of personalized stream filtering to alleviate the “filter bubble” problem and increase the users’ trust in the system.

2 Related work

The theme of this study limits the related work into two areas: the necessity of providing awareness, explanation and control functions in personalized filtering, and existing work on related visualization.

2.1 Necessity of providing awareness, explanation and control in personalized filtering

The goal of a personalized filtering algorithm is to recommend new items according to the user’s areas of interest (content-based filtering) or the opinions of other users with similar liking history (collaborative filtering) (Sarwar et al. 2001). Accuracy metrics, which compare the algorithm’s prediction against a user’s rating of an item, reflect the performance of the filtering algorithm (Mcnee et al. 2006). The underlying assumption is that providing enjoyable items correctly could lead to an increase in user’s satisfaction. A majority work on personalized filtering has been done on improving the accuracy of the filtering models through better modeling user preferences (Gauch et al. 2007).

However, evidences have shown that high accuracy does not always correlate with user satisfaction (Mcnee et al. 2002; Ziegler et al. 2005). A success personalized filtering system should provide useful, not merely accurate recommendations to users (Murakami et al. 2008). An overt focus on accuracy fails to consider the dynamics of user needs and could easily lead to the issue of “filter bubble”: first, pandering to a user’s existing tastes too much may harm a user’s personal growth and usage experience, for example, a user may feel bored if she always reads messages from a given area (Zhang et al. 2012); second, users may not be aware of the personalized filtering that is performed on their behalf, and thus they don’t know what information is filtered and why it is filtered, or even they are not aware that they are seeing a very different set of items than other users see (Resnick et al. 2013).

Providing awareness (letting users know what is hided) and explanation of recommendations (letting users know why it is recommended) can benefit user’s experience in two ways. First, although hided message seems to be irrelevant to user’s existing areas of interest, it has the potential to lead to better user satisfaction. Recent studies on recommender systems have shown that diverse topics and serendipity (e.g., surprisingly interesting content) are favorable (Ge et al. 2010; Zhang et al. 2012). Second, the explanation of why an item is recommended is often welcomed by users. A survey of one movie recommender site shows that 86 % of surveyed users want an explanation feature added to the website (Herlocker et al. 2000). The importance of awareness and explanation on improving user satisfaction (e.g., acceptance, trust) has been extensively discussed (Scheel et al. 2014; Kay and Kummerfeld 2013; Sinha and Swearingen 2002). Herlocker et al. (2000) mention some benefits provided by explaining recommendations such as: justification, user involvement, education and acceptance. In addition, Tintarev and Masthoff (2007a) summarized six motivations for explanations in recommender systems: (1) transparency, which exposes the underlying logic of forming the recommendation so that the user can trust the system; (2) trust, which increases users’ confidence towards recommendations; (3) scrutability, which enables the user to provide feedback on the recommendation to the system, so that the system can improve the future recommendations; (4) effectiveness and efficiency, which helps users to make good and fast decisions; (6) persuasiveness, which convinces users to form target attitude; and (6) satisfaction, which increases users’ willingness to continue use.

Users often desire a more active role in the recommendation process (Xiao and Benbasat 2007). Besides awareness and explanation, allowing users to take control of the filtering process and its outcome is also important (Dooms 2013; Loepp et al. 2014; Mcnee et al. 2003). User’s control can not only provide feedback to the filtering algorithm for adjustment, but also increase user’s satisfaction, as well as persuasiveness of the recommended messages through a sense of control and confidence (Cleger-Tamayo et al. 2013; Tintarev and Masthoff 2012).

In sum, a visualization of personalized filtering which providing awareness, explanation and control could bring mutual benefits to people’s usage experience.

2.2 Visualizing explanations about recommendations

How to visualize explanation about recommendations is important for user acceptance of recommender systems. A visualization should not only present justifications about recommendations, but also consider the interaction with recommendations (Tintarev and Masthoff 2007b). As an early attempt, Webster and Vassileva (2007) proposed an interactive visualization of a collaborative filtering recommender that allows the user viewer to see the other users in her “neighborhood”, who are similar to her, and also to change manually the degree of influence that any of the other users can have on the recommendations of the viewer. A much more elaborated approach of visualizing hybrid recommendation was proposed by Bostandjiev et al. (2012).

Yet all these approaches focus on explaining to the user why she receives certain recommendations and on providing visual tools to change the recommendations. None of them shows the not-recommended items. These non-recommended items remain hidden. Two works take the approach of visually emphasizing items among all other non-recommended items. iBlogViz (Indratmo et al. 2008) is a system that visualizes blog archives. It uses many visual cues to represent the blog content and social interaction history with the blog entry, which help to navigate the blog archive quickly and easily. Particularly, visual cues about the social response (comments) to the news can be used to help users navigate stream data quickly to find interesting news. Rings (Shi et al. 2012) is a visualization of the Facebook social data stream, organized around the people who post in the user’s Facebook stream. It helps the users to browse social data efficiently focusing on the active and influential friends and seeing the hidden time pattern of their social updates, without any filtering. In fact most of the users of Rings used the system to counter-act the Facebook filtering approach and to discover all the posts of their friends that they can’t normally see on their stream.

As discussed above, some approaches for increasing the transparency of filtering process and the users’ trust in recommender systems involve explanations or making the mechanism of recommendations visible to the user. Yet there haven’t been approaches to visualize or explain the filter bubble problem. We propose an interactive visualization that presents a metaphoric view of the recommended and the hidden filtered social data in the personalized stream filtering in an online social networking site. The purpose of the approach is to alleviate the filter bubble problem and increase the users’ trust in the filtered stream.

3 Proposed design for visualization

Our visualization is built as a plugin in MADMICAFootnote 1 system, which is an implementation of an open source social networking framework named Friendica (Nagulendra and Vassileva 2013). MADMICA is built with PHP, jQuery and MySQL technologies. It adds a personalized filtering function to Friendica in order to filter social data. This function was designed according to a model of the strength of the user’s interests in different semantic categories overlaid over a model of their social relationships. Detailed information about the filtering approach can be found in (Tandukar and Vassileva 2012a, b).

The “filter bubble” issue exists in the MADMICA system because the filtering function does not tell users about either what information is filtered or why the information is filtered. Meanwhile, users do not have access to control the output of filtering process. In this section, we propose a visualization that metaphorically explains the filtering mechanism and provides means of control over certain parameters of the filtering for the users.

After users login to the MADMICA system, they are notified in a side menu next to their stream with a message “Do you know this? N posts from your friends are hidden in your news feed based on your interest. Please, click on the bubble below to see them!” This creates awareness to the users that filtering is happening in the stream and some social data are not shown in the stream. When users click on the small bubble icon, the visualization plugin is loaded.

The visualization is based on a bubble metaphor to make the effect of the personalized filtering more understandable for the users (see Fig. 1). It divides the space of the screen in two parts - outside and inside the bubble. The items that are inside the bubble are visible for the user, those outside the bubble are those that have been filtered out (but they are shown in the visualization). The bubble shape design was chosen not only because it fits well with the “filter bubble” metaphor, but also because it is scalable to accommodate more circles inside.

Fig. 1
figure 1

Filter bubble visualization (left: category view; right : friends view)

Full size image

The visualization provides two alternative points of view: one focusing on the user’s friends (friends view) and one focusing on the categories of the social data (category view). The left part of Fig. 1 shows the “category view”. In essence, the category view summarizes what categories of social data the user is interested in (e.g., “News”, “Health”) and what categories of social data she tends to ignore in her stream (e.g., “Movies”, “Sports”).

The right part of Fig. 1 shows the “friends view”. Different from the “category view”, the circles in “friends view” represent the user’s friends who have posted the social data. If a friend’s circleFootnote 2 is inside the bubble, then the social data from that friend are visible in the user’s stream, whereas if the friend’s circle is outside the bubble, the social data from that friend are hidden and not displayed in the stream.

In order to reduce the possibility of overcrowded view and cognitive overload, we follow Shneiderman’s (2001) suggestion: “overview first, details on demand”. Specifically, a list of social updates that belongs to a category/friend is only shown after a user click the circle representing the category/friend (See Fig. 2). For example, by clicking on the “Sports & Games” circle, the user can see all the status updates from her stream of social updates related to the “Sports & Games” category, that have been hidden from her. By clicking each link in the list, the user can see the details of the individual social data items.

Fig. 2
figure 2

Screenshot of hidden posts pop-up window

Full size image

In order to provide a better understanding of what is happening in the personalized filtering process, for both views, the size of the category/friend circle denotes the number of social updates in a certain category or by certain friends, and it helps to understand the relative proportion of social updates that are visible versus those that are hidden as well as who is posting more social data and who is posting less.

Another feature of the visualization design gives users control over the filtering process. This is done by allowing users to drag and drop the category/friend circles inside and outside the filter bubble. Depending on which view is selected (category view or friend view), dragging a category or friend circle inside the filter bubble enables the users to see updates from a category/friend which appears interesting, but so far has been filtered away. In reverse, when users drag a category/friend circle outside the bubble, the social data belonging to that category or from that friend will not appear in the stream anymore. This helps the users to get rid of uninteresting social data and also to avoid spammers who flood the stream with uninteresting and unwanted social data. To let the users know about the results of the drag and drop action, a message is displayed to the user informing about whether the social data will be made visible or hidden based on the users’ action.

Since the filtering mechanism differentiates the filtered data both based on who the data comes from and the category of the data, we added a module (Fig. 3) to allow users to control the display of the visualization. The module contains three parts: bubble view selection, friends/category selection,Footnote 3 and time period selection. If a user selects “category view” and “All” in the friend list, she will see what categories of social data from all friends are visible or filtered. User’s drag and drop actions will show her interest towards different categories. However, a user can specify the filtering of categories from a given friend through the selection of that friend in the friend list. Similarly, if a user selects “friends view” and a category in the “category” list, the user’s dragging and dropping of a friend’s circle will allow this user to permit or forbid the display of social updates under the given category from that friend. The user can also permit or avoid receiving all social updates from a given friend by selecting “All” in the “category” list and dragging the friend’s circle.

Fig. 3
figure 3

A control module on the visualization view

Full size image

In sum, our visualization provides awareness and explanation of personalized filtering process by letting users know the positions of circles (inside or outside of bubble), and users can control the filtering process by moving circles.

4 Study 1: Small-scale qualitative user study

4.1 Experimental setup

A small-scale qualitative study was carried out to understand user perceptions of the filter bubble visualization (i.e., do users really understand what the visualization tries to convey?)

The study was carried out in a lab environment where users were given computers to use the MADMICA system and the visualization. The subjects were 5 undergraduate students from different departments such as public education, public health and statistics. They were recruited through a mailing list of potential subjects for HCI studies. First, the users were given some introduction to MADMICA and then about the “filter bubble” problem. After the introduction, users were given instructions to get familiar with the MADMICA newsfeed homepage and the filter bubble visualization for about 10 min. Once they have explored the system, an interview was conducted. The interview guide consists of a set of questions related to three aspects (awareness of filtering, understanding, control) of meanings that the visualization conveys under the category view and friends view. To answer these questions, users need to interact with the system and think aloud. The users’ voice responses were recorded.

4.2 Methods

The recorded users’ voice responses were transcribed into text and imported into NVivo software, a platform for qualitative research analysis. Thematic analysis (Boyatzis 1998) was carried out to identify users’ correct perceptions and incorrect perceptions of the visualization.

Thematic analysis categorizes qualitative data into themes. It encodes the qualitative information into codes that act as labels for sections of data. The users’ responses were coded and the codes were grouped into three categories, which related to the design elements of the visualization: position of circle, size of circle and drag action. While coding, the number of references for each code was also recorded, i.e., the frequency of that code in the transcript of users’ responses. Then the number of correct perceptions and incorrect perceptions were calculated.

4.3 Results

The thematic analysis results are summarized in Table 1.

Table 1 Thematic analysis results
Full size table
  1. (1)

    Regarding the position of circle visual representation, 108 total references were made. In other words, users mentioned the position of the circles 108 times in all of their responses.

    Users’ Interest is the most correctly perceived category (19 out of 108) about the position of circle. Some excerpts from the transcript for the user’s interest perception category follow: “categories outside the bubble represent the posts that the user doesn’t want to see”, “categories inside the bubble represent my interests”, “categories inside the bubble represent users main interests for the selected duration”, “All the categories outside the bubble represent that none of user’s friends posts are related”, “categories outside the bubble represent the areas outside of my interest for that period”, and “categories inside the bubble represent that the user wants to focus on them”.

    The least referred (2 out of 108) correct perception category regarding the position of circle is relationship. Some excerpts from the transcript follow: “friend circle outside the bubble for a category doesn’t mean that the user unfriended with that friend”, “having some categories inside the bubble for last month for a friend might mean an acquaintance relationship”, and “friend relationship is maintained regardless of user’s friends are outside the bubble”.

    On the contrary, the most referred (24 out of 108) incorrect perception category is the friend’s interest. Following are some excerpts from the transcript: “categories inside the bubble represent friend’s interest and outside represents not interested” and “friend circle more in the middle more interest in the category selected”. Except Interaction with newsfeed (0 incorrect), the least referred incorrect perception is relationship and the excerpt follows: “friend circle outside the bubble represents unfriending”.

  2. (2)

    Regarding the size of the circle, the number of posts is the most referred correct perception (6 out of 16). Users perceive it as follows: “bigger circle for friend/category represents more number of posts and small for less number of posts”. And the second most referred correct perception category is Friends’ Interest, users perceive it as follows: “larger circle for category means the selected friend has more interest on that category”.

    In case of incorrectly perceived category, the most referred (2 out of 16) one is common interest (“small friend circle means more common interest between the user and friends” and “larger circle represents user has less interest on that friend”) and the least referred (1 out of 16) two categories are number of posts and friends’ interest (“small circle means actually posted and big circle means less posted” and “bigger circle outside the bubble represents less interest of friend on that category”).

  3. (3)

    There are two perception categories that emerged by the thematic analysis for drag action: common interest and relationship. Common interest is the most referred correct perception category (4 out of 7). The excerpt from the transcript for common interest perceptions follows: “dragging a category inside means to share more on that category with the friend”, “dragging in may represent my future interest”, “drag out because I don’t want to have common interest”, “drag out means lost interest in that category from that friend”, and “drag all the friends outside the bubble means I want to ignore all the news from them”. Users incorrectly perceived the drag action in relationship category because they thought “dragging outside a friend/category means unfriend”, and in common interest category because they thought “drag inside represents forcing the friend to take interest on that category”.

4.4 Discussion

The results of study 1 suggest that the subjects had both correct and incorrect perceptions about the interpretation of the visualization. Correct perceptions (68 out of 108) regarding the position of circle had more references than incorrect perceptions (40 out of 108). This shows that the majority of users was aware of and had a good understanding about the filtering. In particular, the emergent codes such as common interests, friends’ interest, friends’ sharing, interaction with newsfeed, user’s interest and relationship from the thematic analysis clearly show that the users have some understanding about the filtering.

On the other hand the users sometimes incorrectly perceived the visualization. This could be due to poor graphical language of the visualization and interface as a whole. For example, in the selection module shown in Fig. 3, when “Charlie” was selected in the “Friends” menu and “All” was selected in the Categories menu, users may incorrectly perceive that Charlie’s interests were shown inside the bubble. To solve this problem, an improvement was made to the visualization - the labels were changed into “From Friend(s)” and “On Category(s)” to be more meaningful.

The size of the circle is another indicator for creating awareness about the filtering, i.e., having a bigger size of the circle outside the filter bubble would let the users know that there are more posts that have been filtered out by the system on that category from that friend. Having 75 % (12 out of 16) correct perceptions for size of the circle shows that it is intuitive enough to create awareness about the filtering. The 25 % of incorrect perceptions regarding the size of the circle shows that the graphical language still needs improvement. For example, it would be clearer, if there was a number shown with the varying size. Moreover, the incorrect perceptions of common interest for the size of the circle showed that users may have wrong perceptions about the meaning of the size of circles. For example, the size of the circles was incorrectly perceived to represent the interests of the friends (smaller circle means that the friend has less interest in that category).

The drag action has 57.14 % of correct perception and 42.86 % of incorrect perception. Although the difference is small, considering the number of users who referred the perception can provide clearer indication: the majority of the participants (3 out of 5) were able to understand the control functionality of the filter bubble visualization.

5 Study 2: Large-scale quantitative user study

5.1 Hypotheses

There are at least two limitations in Study 1: first, the small sample size limits the explanation power of the result. And second, users were taught to learn the visualization before the formal experiment, so, the result from Study 1 can’t reflect the intuitiveness of the visualization (whether users can understand the visualization without extra help). Intuitiveness is important because it reflect the ease of use (understanding) when users have no prior experience with the visualization.

Therefore, we conducted a large-scale online quantitative study to evaluate whether the users understand that the visualization provides awareness, explanation and control of filtering. In addition, we aimed to evaluate the intuitiveness of the visualization, as suggested by (Keim et al. 2004). Following business practice (e.g., Google+), we considered providing extra help might be a way to increase intuitiveness. Therefore, we set up two groups: Group 1 (visualization without guided help) and Group 2 (visualization with guided help) and compared the results acquired from these groups. We simply put forward the related hypotheses:

  1. H1

    For both groups, users understand that the visualization provides awareness, explanation and control of the filtering and the filter bubble. And in all, users understand the visualization and its functions;

  2. H2

    Users from Group 2 have more clear understanding that the visualization provides awareness, explanation and control of the filtering and the filter bubble than that of users from Group 1. And in all, Users from Group 2 have more clear understanding about the visualization and its functions than that of users from Group 1.

5.2 Experimental setup

The study was carried out as an online survey, which had the interactive visualization embedded into the survey. Participants were randomly divided into Group 1 or Group 2.

Participants were given some introduction about the MADMICA system and the “filter bubble” problem in general. Both groups of participants were presented with a sample newsfeed homepage embedded in the survey, so that users could actually browse through the newsfeed without leaving the survey page. The sample newsfeed contained around 15 newsfeed items on 5 different categories such as Health, News, Movies, Music and Sports from five different friends named Alice, Bob, Charlie, Dave and Frank. The participants were given instructions to assume that the aforementioned people are their friends in MADMICA system and to browse through the newsfeed homepage as they would do in Facebook. In addition to this, the newsfeed did not show around 7 posts out of those five categories from different friends, i.e., the system filtered out some of the posts. After both groups of participants finished playing with the visualization, both groups were directed to the questionnaire to answer the questions.

5.3 User participation

The online survey was conducted using Amazon Mechanical Turk (MTurk) which is a popular crowd-sourced participant pool. To increase the data quality, we placed attention check questions (ACQs) and restricted participation to MTurk workers with certain qualifications (Paolacci et al. 2010). Each participant’s Human Intelligent Task (HITs) Approval Rate should be larger than 98 % and number of approved HITs should be larger than 5000.

The data collection lasted 1 week and reached our target sample of 400 for both group together. 163 valid responses for each group were collected. For each participant with a valid response, a compensation of 1$ was paid, which is a fairly high rate for an approximately 30–45 min long study on MTurk.

5.4 Questionnaires

We adopted three metrics to reflect understandability. The metrics are adapted based on the International Standards for Software Quality Evaluation (Iso and Iec 2003). Table 2 lists the metrics and their details. Each group was individually tested for their understandability of the visualization. The understandability in Group 1 is measured using the following metrics: Evident Functions, Function Understandability and Understandable Input & Output. Similarly, Group 2 also uses most of the metrics used by Group 1, but the Evident Functions metric is replaced with the Completeness of Description metric because of the help text that is provided with the visualization. We evaluate the intuitiveness by comparing the results acquired from two groups.

Table 2 Understandability metrics
Full size table

There are three independent variables: awareness, explanation and control to jointly measure the overall understandability of the visualization in both groups. Each of the independent variables was evaluated using the metrics given in Table 2. Six questions (2 Yes/No and 4 Multiple Choice Questions) were used to evaluate each of the independent variables.Footnote 4 Altogether, there were 18 questions that were used to evaluate the overall understandability (we use averaged values of three variables to calculate overall understandability).

5.5 Results

We first measured the internal consistency of 18 question items by using Cronbach’s alpha. The Cronbach’s alpha value for Group 1 is 0.700, and for Group 2 is 0.755. The results show that our question items have good internal consistency.

We then assessed the normality of the questionnaire data. The Normal Q-Q (Quantile - Quantile) plot for understandability was generated using SPSS. The results are shown in Fig. 4. Since the data are close to the diagonal line, we assert that the data are normally distributed and suitable for t-test.

Fig. 4
figure 4

Normality Q-Q Plot of Understandability (Left: Group 1. Right: Group 2)

Full size image

A one-sample t-test was conducted. As it is shown in Table 3, for both Group 1 and Group 2, all mean values are larger than 0.5 (0.5 is the average of 0 - does not completely understand and 1 - completely understand). Therefore, H1 is supported.

Table 3 T-test analysis for Group 1 and Group 2
Full size table

An independent-samples t-test was conducted to compare the means between Group 1 and Group 2. The results shown in Table 4 suggest that the mean values of four variables in Group 2 are significantly higher than those in Group 1. Therefore, H2 is supported.

Table 4 Independent-samples t-test for combined Groups
Full size table

5.6 Discussion

The results of study 2 suggest that the users in Group 1 had a good understanding about the visualization. For three individual dimensions of understandability, users have the best understanding about the control (0.7607) of filtering and the filter bubble provided by the visualization. This can be linked with the drag and drop feature of the visualization, which is very popular and commonly used action in many user interfaces and it is a very friendly user interface construct. On the other side, the users’ understanding about the visualization providing explanation (0.6176) to the filtering and the filter bubble has the lowest value (but still larger than 0.5). The overall understandability value of the visualization (0.6967) shows that the users had a good understanding about the visualization after exploring it for the first time without any help and it could be considered as an intuitive visualization.

By comparing the values between Group 1 and Group 2, we find that users in Group 2 had better understanding of the visualization than users in Group 1 in all three aspects. The percentage of increase in understandability of awareness, explanation, control and overall understandability are 12.35 %, 19.87 %, 16.13 % and 15.95 % respectively. This clearly shows that providing some context sensitive help to the visual cues in the visualization could be helpful for users to gain a better understanding about the visualization.

6 Study 3: A field study

In study 1 and 2, we evaluated the understandability of the visualization and we acquired favorable results. However, little is known to users’ acceptance of the visualization. Therefore, we set up a field study to further find out that in real deployment context: (1) whether the visualization increases users’ trust in the filtering; and (2) whether users have good experience with the visualization.

We invited 11 graduate students from our research lab to participate in this field study. They are required to use the MADMICA system instead of Facebook to share interesting and research relevant links over a period of 3 weeks. As suggested by Study 2, a guided help was provided to help users gain better understanding about the visualization. All participants had computer science background and they were very familiar with online social network applications. Six participants were female and five were male.

6.1 Experimental setup

The study took place in a natural environment where users can use their own computers at their own convenient time (like using Facebook). Each user was asked to register at MADMICA and create a profile. Then they added each other as friends and shared anything they found interesting with their colleagues. We provided 11 semantic categories to classify the social data (users manually select the classification when they post any message). The categories were chosen according to the users’ research areas, such as, education & mentoring, user modeling, mobile technologies, social computing, SOA, and common interest areas, such as food & health, news, sports & games, technology, university news and cool stuff. We also allowed users to create their own categories (subject to approval by administrator in the experiment).

We used both closed questions (see Table 5) and open ended questions (see Table 6) to evaluate user’s usage experience and trust towards the visualization. The advantage of open ended questions is: they enable participants to provide free feedback and describe their own ideas or suggestions without any restriction. All of the 11 participants completed the final questionnaires.

Table 5 Closed questions related to users’ trust and usage experience
Full size table
Table 6 Open ended questions related to users’ trust and usage experience
Full size table

In addition to the questionnaires, the usage of visualization of filter bubble was tracked by the system in order to collect usage data about users’ actions on the bubble such as viewing the filter bubble visualization, dragging category/friend circle inside the filter bubble and dragging category/friend circle outside the filter bubble.

6.2 Results

As it can be seen from Table 5, the first subset of closed questions aims at measuring users’ trust in the system. The mean value of each question (from TRU1 to TRU3) is larger than 6. And the mean differences between TRU2 and TRU1, TRU3 and TRU1 are 1.54 and 1.82, respectively. The second subset of close questions aims at evaluating users’ experience with the filter bubble visualization. All mean values (from EXP1 to EXP10) are above 7.

For the question TRU4, Two participants (18 %) responded “yes”, and two said “no”. Seven participants (64 %) answered that their answer “depends”. Among those seven participants, five said that they would trust the system if they had some means to control the filtering mechanism, and other two said they would trust the system if they understood the mechanism.

For the open ended questions regarding usage experience, ten (91 %) participants reported that they used the filter bubble visualization and one participant reported that they did not use it.

(1) For the question EXP11, three participants (27.27 %) commented on what they understood about the category view: “Category wise news/posts” and “I think category view is useful to visualize my choice of posts and help me to somewhat sort the posts I want to have a look on my wall.” The remaining eight participants (72.73 %) commented positively on the aesthetic aspect of the category view (e.g., “nice, compact visualization”, “good, and easy to use”).

Three participants (27.27 %) reported that they didn’t use the friends view. Two participants (18.18 %) said that it’s an unnecessary view and they interpreted it wrongly. Three (27.27 %) reported that it was useful to avoid friends’ social data in which they were not interested. Three participants (27.27 %) said that it was a good and useful visualization.

To a control question asking them to indicate a preference to one or the other view, all of the participants replied that they preferred the category view over the friends view. Five participants (45.45 %) were happy with the current views and didn’t suggest any other useful views. The remaining six participants (54.55 %) suggested several other useful views, such as “a mixture of both”, “more subcategories! But I wonder about the tradeoff with the simplicity”, “time view! Popular view!”, “By Date and week, and popular post -by like and comments”, and so on.

(4) For the question EXP12, nine participants (82 %) stated that they have dragged the category/friend circles from outside the filter bubble to inside the filter bubble. In a follow-up question, those who answered “yes” for dragging inside, were asked about the effect that they noticed after dragging a category/friend circle inside the filter bubble. Eight participants (88.89 %) out of the nine participants said that there is an effect after dragging a category/friend inside the bubble. In particular, four participants out of those eight said that their interest areas expanded and more social data appeared in their stream. Only one participant out of those who tried dragging the circle inside said that there was no effect after the action.

Four participants (36 %) stated that they had tried dragging category/friend circle outside the filter bubble and noticed a change in their stream; particularly social data got filtered away. Other seven participants (63.64 %) stated that they hadn’t tried dragging a category/friend circle outside the filter bubble.

Based on the tracked data, the number of users who performed actions on the visualization, such as clicking on the bubble, dragging a category/friend circle inside, and dragging a category/friend circle outside was plotted for each day throughout the experiment (see Fig. 5). In the first week of the experiment, 19 click actions, 4 drag out actions and 12 drag in actions have been recorded. During the second week when the popup was introduced, the number of click actions has dramatically increased to 28 and while the number of drag outs remained unchanged, the number of drag in actions has doubled as the previous value. In the last week of the experiment, although there is a small decrease in the number of all actions compared to the previous week (21 click actions, 2 drag out actions and 19 drag in actions), the number of actions has increased comparing to the first week.

Fig. 5
figure 5

Number of users performing three types of actions in the visualization

Full size image

6.3 Discussion

The results from this study show that our visualization increases user’s trust towards the system. First, from an open ended question, we know that users are more easily to build trust when they understand or have control over the filtering process. Our visualization perfectly meets user’s needs to build trust because it provides users with awareness, explanation and control. Second, for closed questions (TRU1 to TRU3), users’ trust values differ significantly after they try the visualization. Specifically, users’ trust increases 25 % after they use the filter bubble, and increases 30 % after they see the hidden posts.

Regarding user’s usage experience with the visualization, all mean values of closed questions are above 7 (Table 5), suggesting that user’s usage experience were excellent. Following some user experience design guidelines (Garrett 2010), we consider user experience dependent on whether the artifact is aesthetically pleasing, logically composed and easy to use. 90 % of the participants found that the filter bubble visualization is aesthetically pleasing by rating it above 6; 90 % found that category view was helpful, and 72 % have found that the friend view was helpful. In addition, 72 % of the participants found that the visualization provided adequate awareness about hidden social data, 81 % of participants found that the information on the screen was logically arranged, 63 % of participants said dragging the category/friend circles in and out of the filter bubble was easy, 72 % said finding an interest which is not inside their filter bubble was easy, 81 % said discovering new interests and discovering the interests of friends were also easy and 72 % said that discovering in which areas their friends are most interested was also easy. Moreover, the results showed that users were aware that they are able to find interests outside of their filter bubble and thus discover new interests that they didn’t display otherwise in their behavior. This clearly shows that users became more aware of the filtering mechanism due to the visualization and are interested, able and willing to manipulate it to ensure that they will not be trapped inside a bubble world within the limited boundaries of their manifested interests.

From the results of the open ended questions on user experience, we can see that the category view was more effective than the friends view in creating awareness and understanding of the personalized filtering and also the category view seems to be the most preferred view. This phenomenon occurs probably due to two reasons. First, the option of “Friends view” is listed at the second position in the drop-down list, while “Categories view” is the first option, thus users are more likely to click and use the “Categories View”. Second, it might be easy for users to decide which topics to see in “Categories view”, but hard to choose whose social updates to see for a given topic category in “Friends view”. For example, for a given topic “movie”, it is hard to say Alice’s updates are not interesting but Bob’s updates are interesting. Therefore, the design and logic of “Friends view” needs to be improved. Moreover, the dragging and dropping actions let the participants felt that they had control over their stream and the filtering mechanism.

User’s good experience is also reflected by the results of the user actions graph (see Fig. 5). The graph in Fig. 5 depicts the user actions performed on the filter bubble visualization over the time period of the experiment. The beginning of the graph period can be marked as the learning phase where users get familiar with the drag and drop of category/friend circles. Then there is a sudden spike in user actions in the second week when we introduced a popup window to notify the users that social data are filtered away from the stream and to introduce the visualization allowing them to gain control of the filtering. After 1 week, when the necessary awareness about the visualization has been created, the popup notification was turned off. Even after the notification was turned off, from the graph in Fig. 5, still we could see users checking the filter bubble visualization and dragging the circles in and out. This shows that the filter bubble visualization has been used to control of personalized filtering. Interestingly, most of the actions were “dragging in” categories or people, which means the participants counteracted the filtering mechanism. There were a few “drag out” actions throughout the experiment and they were targeted at one particular participant, the most active one in the group, who was probably perceived as a spammer at a certain moments of high traffic by some of his/her friends.

7 Conclusion

In online Social Networking Sites, filtering of the social data stream is a common way to deal with the overwhelmed social data. However, personalized filtering usually leads to the “filter bubble” problem where the user is trapped inside a world within the limited boundaries of her interests and cannot be exposed to any surprising, desirable information. Moreover, these personalized filters are black boxes, providing no transparency for the user about how the filtering mechanism decides what is to be shown in the activity stream. As a result, the user’s usage experience and trust in the system can decline.

This paper presents a visualization to help users create awareness, explanation, and control of personalized filtering to alleviate “the filter bubble” problem and increase the users’ usage experience and trust in the system. To evaluate the utility of the visualization, three studies were conducted. The first study tested the understandability of the visualization. The second study extended the prior to a larger scale of participants, and tested the intuitiveness of the visualization. And the third study further evaluated user’s acceptance and trust in a field setting. Favorable results are found in all three studies.

However, there are at least three issues need to be discussed. First, the visualization could be technically extended to a richer model. For example, the distance between circles and bubble boundary could be designed to reflect the level of interest. However, we argue that complex design may not always be good for users. A rich visualization may cause several usability issues (e.g., hard to understand, hard to learn, easy to forget and information overload). Even for our simple visualization, the results from both Study 1 and Study 2 show that users still faced difficulties to understand the tool and a guided help is preferred. Second, the “friends view” received controversial opinions from users (e.g., some users thought it was an unnecessary view and it was neglected during usage). But we consider “friends view” is useful because it can help a user easily decide whose social updates can be displayed under a given category. Considering the fact that “friends view” is not as important as “category view”, several operational improvements could be done to make the visualization more acceptable. For example, a button can be added to allow users select default view, rather than listing both views in the selection list. Third, the purpose of our study is to evaluate the utility of proposed design. Both of our questionnaires and experimental design were targeted to the visualization (not the entire system). Therefore, the results of our study cannot reflect users’ acceptance of the whole social network system. However, since our visualization does help users with their usage, it is reasonable to assume that our visualization is beneficial to a social network system with personalized filtering function.

The sample issue is the main limitation of this study. Although the understandability of the visualization has been evaluated in both small-scale and large-scale study, we can only conduct a field study in our lab since a large sample in field experiment is hard to be organized. In our study, the field settings, for example, the scale of subjects (11 students), demographics of subjects, and the length of actual usage of the visualization, are far from enough.

In future work, we would like to continue our research in the following three directions. First, considering the fact that some users might misunderstand our design in terms of the position or size of circles and the drag action, we will try to optimize our design to increase its usability, for example, changing drag and drop action into simple button click action in order to improve ease of use. Moreover, in line with previous studies (Lim et al. 2009), we want to explore the best way to explain the filtering process (e.g., why something is hided, why something is not hided, how to change in current context) to users to increase users’ understanding and control. Second, we will invite more subjects to use our design for a longer period of time. Previous studies have mentioned the impact of prior knowledge on users’ initial understanding of new design (Tullio et al. 2007; Lim et al. 2009), for example, the system provide guidelines to teach users, however, some users do not pay careful attention but reply on their own knowledge. Therefore, we can better evaluate our visualization through long-term usage to reduce the impact of prior knowledge. And third, since our design is an interface of the recommender system in MADMICA system, we will try to utilize user’s drag actions as feedback to improve the recommendation mechanism.