Keywords

1 Introduction

Social media services have become a major source for news [e.g., 1]. Compared to traditional and mostly unidirectional media services (such as printed newspapers or television), these services change the characteristic of distributed information towards being dynamic. Particularly the concept of user-generated content (UGC) implies that users can easily modify information, thus allowing them to add their own opinions or even change the meaning dynamically [e.g., 2, 3].

Consequently, one major disadvantage of social media services and the related UGC is that no trusted authority exists which verifies the quality of information distributed through the services’ networks. For example, it is relatively easy to produce misleading or false information, which is often referred to as fake news [1]. Safieddine etal. (p. 126) [4] express their concerns in this context by highlighting that the idea of online freedom of expression seems to fail: “It has allowed totally unprofessional content; developers bombard predominantly passive web content consumers with news, facts, and stories that cannot be easily challenged.” Placing the focus on social media users, Safieddine etal. observe that they “gradually filter pages, news agencies, or even friends whom they disagree with their political, theological, and/or ethical predispositions” (p. 126), which could lead, amongst others, to a growing number of parallel realities.

Fake news is omnipresent in today’s world and have the potential to cause massive social and monetary damage on every level, i.e., from an individual to a political or societal level [5]. Social media services have become an important instrument during election campaigns since the US election in 2008 [6], and their impact is rising steadily. Two prominent examples of recent political votes, which are discussed, are the Brexit referendum and the election of the president of the United States in 2016. In both cases, fake news was used to manipulate the voters, sometimes even combined with, for example, analyses of social media user profiles [7].

An early fake news detection limits the spread and contributes to trustworthiness of the news ecosystem [8]. However, fake news in social media could mix true with false evidence to support nonfactual claims [9] and create different degrees of fakeness such as half-true, false, etc. [10], increasing the complexity of fake news detection in social media.

Recently, scholars from different disciplines have suggested potential solutions to fight fake news and corresponding damages. From a technical perspective, one promising example is the “right-click authentication” [11,12,13], which allows the reader to easily check with few clicks the source and reliability of pictures posted online. Other scholars see the social media service providers in charge of ensuring true news [e.g., 4]. Complementing these approaches, we place the competency of the individual social media users in the center of this chapter and propose the concept of fake news detection literacy. This is similar to current and ongoing discussions about “media literacy” [14, 15] or “news literacy” [16]. While existing work is mainly of conceptual nature, we offer a concrete way to measure fake news detection literacy.

Our measurement model is based on previous work we conducted to develop a construct to measure the general social media information literacy (SMIL) of a single social media user [17]. In this chapter, we apply and expand the SMIL model according to the context of fake news. We will outline in Sect. 4 that the necessary expansion is mainly relevant for the SMIL sub-category “evaluation”. To sum up, the research question (RQ) of this study is.

RQ: How can fake news detection literacy be measured?

The structure of this chapter is as follows. First, the concept of fake news and its current state of research are briefly outlined. After that, our general SMIL construct is presented. This serves as the basis for the following section in which the SMIL construct is extended and empirically tested with three new fake news-related sub-items. The paper ends with a discussion of implications for research and practice and potential applications of the fake news detection literacy model.

2 Fake News

When it comes to the consumption of news, social media services have outpaced traditional sources such as paper-based newspapers or television formats [18]. While social media services on the one hand can offer a more convenient and tailored customer experience, they on the other hand build the basis for fake news. For example, news feeds in social media typically contain public as well as private postings and are intertwined with the online activities of the consumer [19]. This, amongst others, makes it very difficult for the consumer to evaluate the quality of the news.

While fake news as a term is widely adopted, its academic definition is subject to intense discussions [20]. Starting with a very generic definition, Allcot and Gentzkow (p. 213) [1] describe fake news as

news articles that are intentionally and verifiably false, and could mislead readers.

Gelfert [20] employs a broad and deep discussion of the term fake news. He identifies several similarities among extant definitions. First, the medium internet, particularly social media, plays an important role for both creation and dissemination of fake news. Second, fake news do not have any factual basis. Third, fake news are intentionally misleading. He compares and criticizes extant definitions (which is not further considered in this chapter) and suggests his own one:

Fake news is the deliberate presentation of (typically) false or misleading claims as news, where the claims are misleading by design. (p. 108)

The main formats of fake news are images, videos, and text [4]. Furthermore, scholars have elaborated that there are some general characteristics of fake news such as the content, user response, source, and spreaders [21]. Different types of fake news and fake news related terms (gossip, rumors, satire, etc.) show various forms of these characteristics [20,21,22]. More details about the key characteristics as well as linguistic analyses and user engagement studies of fake news´ properties will be discussed in detail in Sect. 4.1.

Considering the supply side, there seem to be two main motivations for producing fake news [1, 20]: The first one is financial reward. It is possible to draw substantial advertising revenues from clicks on the respective site. A popular case is the one of teenagers in Macedonia who earned thousands of dollars with produced fake news about both Clinton and Trump during their election campaigns [1]. The second motivation is ideological. Taking again the example of political elections, producers of fake news try to advance the politician they favor.

We believe that technical solutions to fight fake news, such as the “right-click authentication” [11,12,13], are a step into the right direction, but even more important are literate social media users. Literate, or competent, social media users are more likely to detect fake news. This means they are more critical regarding the reliability of news, and they are willing to spend time on conducting required proofs. We call this literacy “fake news detection literacy”. It is strongly linked to social media information literacy (SMIL) and, thereby, to the general topic of digital competencies [23]. However, in order to answer further research questions, for example, about the relation of individual fake news detection literacy and actual fake news detection, a measurement model of individual fake news detection literacy is required.

In a previous paper [17], we have developed a measurement model of SMIL, which is briefly summarized and explained in the next section. Taking this as the basis, we will then suggest an expanded measurement model that is applied to the context of fake news detection.

3 Introduction of SMIL

3.1 Development of SMIL Definition

In our previously mentioned research article, we developed a new construct from scratch, which is meant to measure a user’s information literacy regarding social media (SMIL). The article is theoretically motivated and based on MacKenzie etal. [24] who provide a detailed step-by-step guideline for construct development in general. Our main argument for choosing this approach for the core SMIL construct was the possibility of starting at an initial stage for such a core construct, rather than relying on an existing construct that is expanded, but not completely suitable. SMIL itself in its current state is a suitable basis for expansions such as fake news, though, because it regards the specific elements social media bring along, especially its dynamic processes [18, 25].

MacKenzie etal. [24] describe the construct development process in several steps, which we applied for SMIL consecutively. Based on the guidelines of Webster and Watson [26], we conducted an extensive review of existing literature in the field of social media research. Scanning various scientific databases like Scopus, ScienceDirect, JSTOR, EBSCO and others was a key element due to the fast-changing environment, with new social media services being on the rise and providing new functions to its users [18]. Regarding particularly the perspective of literacy in the context of social media, we applied the following search query:

information literacy AND social media OR construct * OR measure *

Starting with in total 88 articles which cover a rather wide range of academic fields and aspects of literacy (i.e., metaliteracy, transliteracy [27], or reading literacy [28]), we extracted relevant keywords in multiple iterations. We then clustered these keywords according to the description of MacKenzie etal. [24] to create a holistic definition of SMIL (Fig. 1).

Fig. 1
figure 1

Definition of social media information literacy (SMIL) [17]

Full size image

3.2 SMIL Item Development, Evaluation and Refinement

Based on the SMIL definition, the next couple of steps in MacKenzie etal.’s [24] guideline recommend the derivation of individual items. This includes their phrasing as well as their initial testing and refinement. By literature screening, we extracted a portfolio of existing items from academic sources which we then completed with new items derived from our SMIL definition. This led to a total of 40 unique items that were each associated with one of the clusters of abilities that form our SMIL definition.

The item evaluation was then applied following the quantitative approach of Hinkin and Tracey [29]. We calculated the items’ content validity with the help of results from 59 surveys that we conducted. This outnumbers the threshold of 50 which the authors suggest. With the method of one-way repeated measures ANOVAs, we tested how statistically significant the items within the cluster are. Consequently, we rephrased 14 items according to Wieland etal. [30] to increase content validity.

Whereas our complete step-by-step procedure following MacKenzie etal. [24] is explained in detail in our original SMIL article [17], we briefly summarize the results of the final refinement step, which ultimately formed the core SMIL construct. We enlarged our empirical setting to conduct reliability checks by spreading the survey to a different target group from a different country to includecross-cultural perspectives. Additionally, we capitalized on a crowdsourcing campaign which gave us insights from an even more heterogenous number of participants. The sample was large enough to evaluate our 40-item scale with an exploratory factor analysis (EFA) based on participants’ item ratings. Results based on the Eigenvalues revealed an optimum of seven to eight clusters which corresponds with the number of eight abilities that constitute our SMIL definition. Within the eight clusters, we eliminated in a final refinement process those items with low factor loadings or cross-loadings.

The following Table 1 presents all 40 items associated with the eight SMIL clusters, of which 19 form the core SMIL item set. The remaining items shaded with a dark grey background were eliminated from the first version of the item set and, thus, the entire initial version of the SMIL construct after the aforementioned validity tests.

Table 1 Overview of 40 SMIL items
Full size table

Especially items of the clusters obtain information and understand information were deleted due to the factor loadings. For evaluate information, we could identify two separate factors accounting for the cluster of which one addresses rather abstract evaluation and one refers to concrete action of evaluation accomplished by users. Beside the content-related motivation to expand the core SMIL construct towards fake news, this split of evaluation clusters reinforces the decision also from an empirical perspective. Because within evaluate information, the items Eval_5 and Eval_6 are directly linked to the evaluation of faked information and rumors. Thus, we proceed in the next section with the application of the core SMIL construct to the context of fake news by primarily expanding EVAL_5.

4 Applying SMIL to the Context of Fake News Detection

4.1 Key Characteristics of Fake News

There are three generally agreed upon characteristics of fake news: its content style, the user engagement with it, and the source users publishing it [21].

The semantic characteristics of fake news content vary across different types of fake news and fake news related terms such as gossip, rumors, hoaxes, satire and etc. [20,21,22]. However, scholars focusing on text analysis of fake news have found some linguistic cues regarding pronouns, conjunctions and word patterns [21]. Fake news contains personal pronouns and words associated with negative emotions such as swearwords [31]. High uncertainty or many typographical errors are other cues for news content of low quality [32]. In addition, low quality and high informality of the headline are two characteristics of fake news [33]. For instance, in a high quality news article there is a similarity between the headline and the body-text [22]. However, swear words (‘damn’), net speaks (‘btw’ and‚ ‘lol’), assents (‘OK’), non fluencies (‘er’, ‘hm’, and, ‘umm’), and fillers (‘I mean’ and ‘you know’) are signs of informality in the headline [32].

The second characteristic is the emotional response that news generates. Fake news contains opinionated and emotionally provoking language generating a sense of confusion [21]. Furthermore, sensational or even faked visual impressions (e.g. images and videos) can be employed to provoke specific emotional responses from consumers [34].

The third characteristic of fake news is the source promoting it. Starting from the URL structure, the source media and the author of the news, different properties of a publishing source should be examined [21]. Corresponding with the definition of fake news suggested by Gelfert, sources and channels promoting fake news are misleading “by design”, to manipulate the audience’s cognitive process [20]. For example news posted on an unreliable platform and promoted by unreliable users is more likely to be fake news than news published by authoritative and credible spreaders [32].

4.2 Expansion of the SMIL Core Construct

The described characteristics of fake news ultimately lead to the expansion of the core SMIL construct. Expanding the construct is advisable because of the dynamic environment fake news predominantly appears in, which is social media as previously characterized. The specific expansion is visualized in Table 2, with modifications precisely originating from the closest item of the core construct, EVAL_5.

Table 2 Fake news expansion of the core SMIL construct
Full size table

The expansion of EVAL_5 is the major change we propose regarding the application of our SMIL construct towards fake news detection. This core item of SMIL is directly linked to the topic and we differentiate between three new aspects that shed more light on the evaluation ability of a social media user regarding fake news. We call these aspects sub-items, i.e., EVAL_5 – SUB_1 to EVAL_5 – SUB_3, which are meant to represent a hierarchical graduation between the existing superordinate core item and the new ones dedicated explicitly to areas of fake news.

Whereas the headlines itself can sometimes help revealing fake news based on quality and formality on their own [33], the interplay between the headline and the main text-body is an even stronger criterion for evaluation. Especially the mix between a headline that is intended to raise awareness by capitalizing on clickbait elements and a text-body that refers to rather accurate content is important [35]. Consequently, EVAL_5 – SUB_1 addresses this ability. The second sub-item EVAL_5 – SUB 2 refers to a different aspect, the ability to distinguish between truly fake news and satirical content with partly similar patterns, but also recognizable differences [21], e.g., the motivation for spreading the news. The third new sub-item EVAL_5 – SUB 3 represents the ability of a social media user to identify whether the source of an information is a real person or only an automated system. Bot networks are the most common thread in terms of fake news, which is specifically regarded with this sub-item [36].

Besides evaluation, other clusters of this core construct are also affected. Particularly those dealing with the quality of information sources, i.e., items SEA_5 (appropriate choice of information sources), UND_2 and UND_4 (interplay between multiply sources). Additionally, the remaining items of re-evaluation can be of relevance in terms of fake news identification. The exchange with and reactions from other users can indicate a previously not recognized fake news appearance. We do not see the need to state particular sub-items for these core items. But while applying the SMIL construct in the context of fake news, including the new sub-items within the evaluation cluster, special attention should be paid to these items of the core construct as well.

4.3 Empirical Evaluation of the Extended Model

In a next step, we empirically tested our model extension towards fake news. For this, we used Amazon Mechanical TurkFootnote 1 (MTurk) to collect data. MTurk has gained popularity among researchers [37], as it enables both to conveniently access a large pool of potential respondents, and to receive responses without any time delays at comparably low costs (particularly compared to paper-based study designs). However, aside from these advantages, it must be noted that the quality of the responses is a critical aspect. Thus, as suggested in related literature [38], we integrated two direct quality checks in our survey. First, we implemented a captcha questions which had to be answered before the survey begins. This question was designed as a simple calculation, e.g., 37 + 3 = ___, that nevertheless requires human knowledge, and therefore does not allow for instance bots entering the survey. Second, we implemented a test question in our total set of question which reads as follows: I am able to develop the next Facebook. If you are reading this question, select I completely disagree. Answers different from I completely disagree on this question led to an exclusion of the respondent from the dataset, as it can be assumed that questions were not read.

In addition to these two direct checks, we considered the time to fill the questionnaire by a respondent (‘input time’) as an indirect check. Based on test runs, we defined 2:30 min as the minimum input time that is required to enter meaningful answers. For questionnaires with less than 2:30 min input time, a high chance for random clicking by the respondent is given. We therefore eliminated these respondents from our dataset.

The initial dataset for the empirical analysis consisted of n = 172 completed responses at the time of survey closure. Based on the aforementioned exclusion criteria, we reduced the sample to n = 96 responses because 59 participants did not reach the time threshold, additional 16 did not respond to the test question appropriately and one respondent claimed to be 6 years old, thus underage. A majority of the 96 respondents stated that they are from the USA or India, approximately 58% stated that they are male (39% female, 3% did not disclose a gender). The average response time for the survey including the specific fake news extension of the research model was 4:45 min (sd = 2:08).

All 40 core items and the three additional fake news items were measured on a 5-point Likert scale ranging from 1 (I completely disagree) to 5 (I completely agree). Thus, we can assume equidistance and calculate the average mean for the three fake news items. The highest score of 4.52/5.00 was measured for EVAL_5– SUB 1, revealing a strong confidence within the sample of being able to differentiate between a news headline and its text-body in contrast. Results for the remaining two items also indicate rather strong confidence in the respective abilities, but are lower for EVAL_5– SUB 2 (4.16/5.00, “ability to distinguish between satire and fake news”) and EVAL_5– SUB 3 (3.61/5.00, “ability to identify automated accounts/bots”).

Our results indicate both similarities between the three fake news items, as all have above-average means, but also differences because these means still tiered. Thus, we proceeded with a correlation analysis using IBM SPSS Statistics 25 to investigate the relation between the new model items. We calculated bivariate Pearson correlation coefficients and tested for two-tailed significance (Table 3).

Table 3 Correlation analysis between the three fake news-related sub-items
Full size table

Our results demonstrate a strong positive correlation that is highly significant between EVAL_5 – SUB 1 and EVAL_5 – SUB 2 in the sample. It suggests that social media users with the ability to identify differences between a news headline and its text are also able to differentiate between satirical news elements and those that are faked. However, these users not necessarily have the ability to recognize automated news created by bots as there is no significant relation between these sub-items. Although weaker, a statistically significant correlation exists as well between EVAL_5 – SUB 2 and EVAL_5 – SUB 3. This indicates that social media users with a higher ability to distinguish between satire and fake news likely are able to identify bots.

In summary, our empirical correlation analysis’ results show that the three fake news-related sub-items of the core item EVAL_5 are significantly linked with each other. This supports our initial claim that the core SMIL model should be extended if the fake news phenomenon is to be analyzed more precisely. We suggested three new sub-items which are statistically related with each other and which might serve as standard items for future empirical fake news studies based on SMIL.

5 Contributions, Limitations, and Future Work

The suggested model to measure fake news detection literacy is of value for both researchers and practitioners. Considering research, we contribute to the field of the digitization of the individual and corresponding micro-foundations, as we do not investigate a digitalization phenomenon on the organizational but the individual level [39]. Our model establishes a link to previous theoretical work about rigor construct development that originates from scratch [24], it is based on the already initially tested core SMIL construct [17], and it allows various other applications linked to fake news or related topics within the social media environment. The initially raised research question can be answered with the newly introduced sub-items, which are derived from theory and turned out to be meaningful considering our empirical testing (see Sect. 4). Thus, fake news detection literacy can be measured by adding the three sub-items to the original core SMIL construct and by specifically regarding the discussed core items of the search, understand, and re-evaluate cluster.

Practitioners however can benefit from a concrete set of items, which allows them a hands-on approach towards news classification. We provide them with our expanded SMIL construct, an instrument that regards the dynamic changes in the fast-paced social media context. One example that demonstrates the value of the expanded item set would be the concepts of competency and literacy, which are of growing importance in the educational system. Curricula are often designed to impart competencies but not pure content, which also corresponds to the requirements formulated by accreditation agencies. Thus, when curriculum designers and teachers could assess the fake news detection literacy of pupils or students, the development of respective courses and materials could be more focused and tailored. This also applies to other (commercial) training providers, covering different age groups and subjects. Human resource managers is another stakeholder group for our model. Similar to educational institutions, companies might also be interested in the fake news detection literacy of their employees in the age of digital information. Information has become a critical resource for many businesses, which underscores the importance if literate employees in this regard.

Considering the limitations of our study, we are aware of the fact that our empirical assessment of the three added sub-items (see Sect. 4.3) should be interpreted as a first step towards validation. In upcoming studies, the newly derived sub-items could either replace the EVAL_5 core item entirely or could be added to the core item set again to replicate our study design. Going beyond our design, empirical causality could be tested, e.g., with an explorative and confirmatory factor analysis, as it has already been performed for the core model by [17].

Another more general limitation of our study is the assumption that self-assessment is a suitable approach to measure literacy or competency. We are aware that self-assessment is always at risk of bias, and therefore we vote for combining it with alternative approaches such as experiments or observations. However, our set of items could also serve as the basis for other approaches such as interviews (e.g., our items could be used to develop an interview guideline).

Aside from this general view on future research opportunities, we have identified two more specific application areas for our fake news detection literacy model. First, the postulated positive relationship between fake news detection literacy and actual fake news detection performance should be investigated. On the one hand, we believe that it is impossible to identify every piece of fake news, on the other hand we believe that a social media user with a certain level of fake news detection literacy should be able to identify most fake news. A corresponding research question could be What is the necessary level of fake news detection literacy? Following this line of argumentation, it may not be useful to aim for the highest possible level of fake news detection literacy. Instead, it could be more important to focus on dynamic training approaches, which brings us to the second specific avenue for further research. The overarching question here would be How can we impart fake news detection literacy? This question is not trivial to answer, given the extremely dynamic and innovative field of fake news production, which often uses text mining and other big data analytics to provide the consumer with the “right” fake news. Another related question is the one of responsibility. Who is in charge of imparting fake news detection literacy, or, on a general level, social media information literacy? Is it the teachers, who often lack these literacies themselves? Or is it the parents? Or is it the employers? Or is every individual user responsible for his- or herself? Given this complex setting, we believe that this topic requires interdisciplinary research efforts particularly from the fields of information systems, psychology, and education.

In the age of digitalization and information, fake news can cause massive damages to an individual person, to a company, or to an entire society. Empowering people with the necessary competencies, more precise with fake news detection literacy, is therefore a key challenge and we believe that our measurement model marks a valuable contribution towards the next level of understanding.