Fake or Fact? Theoretical and Practical Aspects of Fake News

Abstract

The phenomenon of fake news is nothing new. It has been around as long as people have had a vested interest in manipulating opinions and images, dating back to historical times, for which written accounts exist and probably much beyond. Referring to it as post-truth seems futile, as there’s probably never been an era of truth when it comes to news. More recently, however, the technical means and the widespread use of social media have propelled the phenomenon onto a new level altogether. Individuals, organizations, and state-actors actively engage in propaganda and the use of fake news to create insecurity, confusion, and doubt and promote their own agenda – frequently of a financial or political nature. We discuss the history of fake news and some reasons as to why people are bound to fall for it. We address signs of fake news and ways to detect it or, to at least become more aware of it and discuss the subject of truthfulness of messages and the perceived information quality of platforms. Some examples from the recent past demonstrate how fake news has played a role in a variety of scenarios. We conclude with remarks on how to tackle the phenomenon – to eradicate it will not be possible in the near term. But employing a few sound strategies might mitigate some of the harmful effects.

1 A Brief History of Fake News

To be clear, fake news is nothing new. Intentionally circulating rumors and false information have been around for as long as humans have realized its influential power. What makes fake news seem new today, is the medium, speed and precision with which it can be distributed on a much larger scale by micro-targeting particular users. The term fake news has gained significant prominence during and after the 2016 US general election. This was partially due to BuzzFeed News’s story on teenagers in the Balkans who had intentionally spread false information in support of Donald Trump [13]. Their investigation revealed that at least 140 US politics websites were created and run by young Macedonians who had used Facebook adverts to spread fabricated news articles in favor of Trump. Their ultimate goal was not to help him win the election but to make money by generating lots of traffic to their websites [55]. According to several teenagers interviewed by BuzzFeed, Facebook was deemed as the best platform for spreading fake stories [55]¹. As false information and rumors have been around for centuries, the medium with which they can be disseminated today, namely, via social media, has made fake news look like a novel phenomenon, but, in fact, fake news is anything but new.

It is challenging to precisely pinpoint the origins of fake news. Following the definition of Allcott and Gentzkow [5] as “news articles that are intentionally² and verifiably false, and could mislead readers,” it could be argued that fake news has its origin somewhere after the invention of the printing press in 1439, which allowed news to be printed and thus shared on a much larger scale as in contrast to the preprinting press era, in which news spread from person to person. Soll [58] supports this argument by claiming that “Fake news took off at the same time that news began to circulate widely.” As there were plenty of news sources available, including official publications by political and religious authorities, a concept of journalistic ethics³ or objectivity was lacking, he argues [58]. This, in turn, enabled fake news to be distributed for various purposes, such as political or economic gains. One of the earliest examples of using fake news for profit was the often-cited example of the New York Sun in 1835, which ran stories about a British astronomer who had discovered, among other things, “blue-skinned goats” and “giant bat-like people” by using a newly invented telescope to view the moon. This story led to a massive surge in sales, and for a short time, The New York Sun became the world’s bestselling daily paper thanks to intentionally spreading fake news [60].

In a similar vein, Leetaru argues that fake news is as old as the news itself, but goes further and states that it became an inherent part of wartime propaganda, especially during World War I and II [38, 39]. Leetaru bases his claims on an experiment with Google Books NGrams Viewer – an online search engine that plots the frequencies of words and phrases using a yearly count of n-grams found in printed books between 1800 and 2000. Using this search engine, Leetaru was specifically interested in how often the term fake news had appeared in books in the last two centuries [32]. He found the term was initially used at the start of World War I and reached its peak just before World War II. He explains the peak by noting that it is likely reflecting the rise of research into propaganda and its potential impact on societies [38, 39]. All in all, he argues, the phrase fake news originated somewhere around the beginnings of WWI and WWII but came to rapid prominence after the 2016 US general elections.

Others, such as Burkhardt, point to the preprinting press era as a new and potential origin of fake news⁴. However, Burkhardt carefully caveats there are no means to verify these claims. She continues by explaining that information was inscribed on materials, such as stone, clay and papyrus and was usually limited to the group leaders, who ultimately controlled this information. This control, in turn, gave them power over others as group members heavily relied on knowledge or information obtained from the leader [39]. So, intentionally spreading fake news to smear other group leaders, for example, could have been possible and would have probably worked, because “without the means to verify the claims, it’s hard to know whether the information was true or fake news” as Burkhardt argues [12].

Numerous examples of fake news can be found throughout the post-printing press era. American writer Edgar Allen Poe (1809–1849), for example, could be considered a fake news author as he is credited with writing at least six stories that all turned out to be hoaxes, most famously, the story of a balloonist who had crossed the Atlantic in 3 days [9]. The emergence of fake news is not limited to a specific century or period of time. In fact, numerous examples are documented, ranging from the mid-fifteenth century up to the present day.

As the means to communicate have developed, so has the way in which fake news has been circulated too. In the mass media era [12], for example, when radio broadcasting became widely available, fake news had also been around. One famous example is The War of the Worlds broadcast in 1938, which was a narrated adaptation of H. G. Wells’ novel The War of the Worlds. Even though the presenter emphasized at the beginning of the show that the following broadcast was a narrated novel, a mass panic followed as many people confused the novel with real news [68].

In sum, it can be stated that fake news has always been around for as long as humans have existed and realized its power of influencing others for economic, social, and political gains. It is still unclear when it emerged for the very first time, but many historians, journalists, and scholars point to the invention of the printing press, which gave rise to news media outlets and eventually to fake news. Numerous examples can be found throughout history, leading up to the 2016 US general elections and due to the pervasive nature of fake news, it is highly likely that rumors and false information will remain with us for the foreseeable future.

2 Why People Fall for Fake News

One reason why people fall for fake news is the fact that they perform poorly at fact-checking. Research from cognitive psychology has demonstrated that people are naturally bad fact-checkers and comparing known to unknown issues poses a significant challenge [29]. The so-called Moses Illusion, also known as the semantic illusion, serves as an illustrative example. This phenomenon was first identified in 1981 in a study that examined how meanings of individual words are combined to form a more global description of meaning [29]. The Moses Illusion occurs when people answer “two” in response to the question “How many animals of each kind did Moses take on the ark?” even though they know, it was Noah and not Moses who took animals onto the ark. The study found 81% of the participants (N = 27) did not notice the error in the question although they knew the correct answer [27]. Psychologists call this phenomenon knowledge neglect [40, 41].

So, why are humans so bad at noticing errors? There are several explanations for this, most of which stem from psychology. According to [29], two peculiar habits of psychology make humans miss these errors. First, their general bias to initially process and label new information as true, also referred to as confirmation bias [31]. Second, as long as new information is perceived as close to the truth, it will automatically be accepted. The underlying idea refers to the good-enough model, which presumes that to maintain conversations, “humans accept information that is good enough and just move on” [31]. In other words, small errors in sentences will be accepted in order to keep the conversation flowing.

Another reason why people believe fake news stories is a particular cognitive shortcut that occurs when users have to decide whether or not to share a story on their social media feed [14]. In general, the human brain uses cognitive shortcuts or simple heuristics⁵ to avoid information overload when making a decision. These shortcuts or heuristics are intuitive, short, and simple rules that subconsciously issue instructions as to what to do. In this respect, a research study examined the hypothesis that negative news stories are more likely to be retweeted on Twitter as in contrast to positive, nonnews tweets. The findings suggest that indeed, negative sentiment enhances virality in the news segment (but not in the nonnews segment) [34]. To put it simply, people tend to retweet negative news headlines without even challenging their credibility [14].

A further factor contributing to why people fall for fake news are so-called echo chambers [14]. An echo chamber in the context of media is a filter bubble around a user, in which a person is exposed to content that amplifies or reinforces existing beliefs. As users are free to choose whom they want to follow, or be friends with, or which web-sites and sources to read, it is up to the individual to decide what content they wish to see on their social media feed. The danger here is that users who only view content that reinforces existing political or social views are more likely to believe fake news, because people generally tend to favor information coming from within their social circles [20]. These social circles, however, can act as an echo chamber [53]. Therefore, users will most likely not challenge the authenticity of the news article or the source as they already trust their social circle (following a source-based trust model in that sense) [20].

In a longitudinal research study, researchers subjected the Facebook activity of 376 million English-speaking users to critical scrutiny by examining how Facebook users interacted with English-speaking news sources [54]. Their findings demonstrate that the higher the activity of a user, the more the user tends to focus on a small number of news outlets, which in turn, leads to distinct online community structures and strong user polarization. In other words, social media creates an echo chamber that provides the ideal environment for fake news to spread. In a similar vein, the authors conclude that the polarization of users online is probably the main problem behind misinformation [54].

Another reason why people fall for fake news, especially when it comes to images, is a humans’ inability to detect photo forgeries. Considering that 3.2 billion images are shared each day on average [11] and highly sophisticated photo-editing tools are widely available, photo forgeries constitute a significant challenge. Images can sometimes form a critical part of a fake news story, making research in this area paramount. However, according to Nightingale et al. [45], there is a lack of research which specifically examines humans’ performance in detecting photo manipulations. For this reason, Nightingale and colleagues conducted one of the first experimental studies of its kind, in which participants were presented both original and manipulated photos of real-world scenes to classify these accordingly and point out the manipulations. The results indicated that humans have an “extremely limited ability to detect and locate manipulations of real-world scenes” [45].

Communication and the sharing of information is known to be a key factor for the use of social media. Whereas the sharing of correct information may foster a better-informed society, the sharing of inaccurate and misleading information may have negative consequences [37]. Information quality – regarding the accuracy, truthfulness, or veracity of information on a particular medium or perceived information quality, an individual’s perception of these factors – play a fundamental role in the sharing behavior of users and thus have a strong impact on the dissemination of fake news. High perceived information quality may encourage the sharing of messages whereas low information quality may lead to reduced willingness to share and increased consciousness about one’s online reputation.

In sum, human psychology, communication patterns, echo chambers on social media, the inability to detect photo or video manipulations, and the perceived information quality of platforms are all critical factors that enable fake news to flourish. The good news is, however, there are techniques to spot fake news! But this will ultimately be up to the individual to decide whether or not he/she acknowledges the existence of false information and rumors on social media. Most importantly, individuals will need to be ready to learn basic techniques to assess and verify the credibility of the news article they read. Efforts to increase digital literacy and use of social media in education and training, on all levels from governments to companies, platform providers, news-outlets, and individuals, should complement such efforts and ultimately lead to a more conscious manner of news consumption and distribution behavior.

3 Fake News: Practical Aspects

If you would ask someone what “fake news” was 10 years ago, they would probably indicate towards humorous satire websites such as The Onion ⁶ or The Daily Mash ⁷. Today, the definition of fake news is more nuanced, more diffuse, and a lot less humor-oriented and includes a wide spectrum of types of publishers and content distribution models.

The simplest categorization of fake news would be split into “mis-” and “dis”-information. Misinformation, which is not intentional and disinformation, which is intentional. Both are used to manipulate and influence opinions and can have a serious impact on topics and segments of society. Oftentimes, accurate and inaccurate facts and information are intertwined and twisted to provide slanted viewpoints and half-truths mixing fact and fiction [28]. The wide adoption of the Internet and of social media platforms, in particular, have made it very easy to create, distribute, and consume content in a rapid and massively parallel manner.

Misinformation, whether generated by unreliable sources of information or low-quality journalism, can have a significant impact on media consumers. Two relevant examples detailed further in this chapter, such as wrongly accusing BMW of rigging car emissions in the manner of Volkswagen, or ABC News causing the Dow Jones to fall by publishing a report wrongly accusing candidate Trump of colluding with Russia, demonstrate just how important it is to maintain a high-quality information verification process within any editorial newsroom.

Disinformation can take many forms and covers a wide spectrum of fake news: from the seemingly innocent satire websites to clickbait, source hacking, sponsored content, conspiracy theories, and even state-sponsored propaganda. What disinformation aims for is a significant impact in the information space, which can be obtained almost exclusively in the online battlegrounds where audiences are easy to reach and even act as content distributors themselves, while interacting with the content.

4 Tackling Fake News

Analyzing the most recent prominent fake news scandals, from the Indonesian presidential elections smearing campaigns in 2015, the 2016 US Presidential elections, and the ongoing conspiracy theories that once in a while get published by mainstream media, recurring patterns into the techniques used to create fake news with most impact can be identified. These patterns have since been structured into identified signals presented in digital literacy programs and manuals (such as the ones put together by the News Literacy Project) and also implemented into algorithms used in academic or commercial software applications designed to spot online fake news, such as Hoaxy ⁸ developed by the Indiana University Open Networks Institute or TrustServista ⁹ developed by Zetta Cloud.

The fake news scandal that erupted after the 2016 US presidential elections has put a spotlight not only on social media platforms, such as Facebook or Google, accused of not taking action against the spread of disinformation but also on researchers and journalists, who were expected to bring expert-solutions to the problem. The main directions of addressing online fake news, mainly since late 2016, can be categorized as follows:

• 1. Professional Fact-Checking, when information is verified by media professionals. This approach provides an accurate and standardized method of information verification, which eliminates any bias and can lead to identical results even with different impartial verifiers. Notable examples of such professional fact-checking groups are Snopes.com , First Draft News or FactCheck.org . The disadvantage of this approach is that it is very time-consuming, it does not scale (considering the limitless size of the Internet), and most of the time it is a sluggish “post-mortem” analysis performed after the viral fake news content was distributed and has reached its goals.

• 2. Crowdsourced Fact-Checking, when verification is performed by non-professionals on virtual online platforms, using commonly agreed principles and standards of work. The idea of this approach is to be independent, to leverage “crowd wisdom” and to create a scalable model. In reality, it proved less successful than the professional fact-checking approach, with only a few such initiatives becoming known to the public (Reddit, WikiTribune), and results being challenged for being potentially partisan or being derailed by strong online trolling groups such as user boards 4Chan.org or 8Chan ¹⁰.

• 3. Automated Algorithmic Verification, when content verification is performed in an unassisted, automated manner by a software program. This approach is being used, in combination with human review, by Facebook and Google, and also AI startups such as Zetta Cloud, Factmata or Unpartial. The idea behind this approach is to successfully filter out most of the untrustworthy online content that matches certain statistical patterns, with potential to work in real-time, at scale, and a “good enough” fake news detection quality, similar to those of email spam filters.

The fake news types with most of the impact in the online space – clickbait, conspiracy, and propaganda – have common “red flags” that can be used to identify them as untrusted content, even if the content is in video, image, or text format. These “red flags” make up identifiable signals that a human content consumer with moderate digital literacy or critical reading skills could use; also, signals that could be implemented as software algorithms for the automatic detection on online disinformation, in real-time.

5 Algorithmic Fake News Detection

One of the areas of interest with regard to curbing the fake news phenomenon is the research of automatic fake news detection using Artificial Intelligence algorithms. Since the 2016 US presidential elections fake news scandal, more effort has been put into researching and developing tools that can detect fake news stories in the online space.

One such tool was launched in early 2017 by Zetta Cloud ¹¹, a Romanian AI startup that received a grant from Google to develop a software prototype that can automatically detect fake news. The tool called TrustServista ¹² was used for publishing a “News Verification Report” [63] detailing an approach to standardizing “red flags” for automatically detecting fake news with no human intervention. The report analyzed 17,000 online news articles (collected from news agencies, newspapers, blogs, magazines, and organization websites) in order to find patterns into how the quality, source referencing, and authorship of online news impact the trustworthiness of the produced content, outlining the current Text Analytics and Semantic Reasoning capabilities for the algorithmic detection of such content.

The final metric according to which the content was analyzed is called a “trustworthiness score” and indicates if a story is trustworthy or not, rather than classifying it as true or false or performing any checking of the facts found in the article. The trustworthiness score takes into account the following information:

• Writeprint – Source and Author identification:

  • Is it published by a known established publisher?

  • Can the publisher’s credentials (ownership, editorial team) be verified?

  • Does the article have a named, verifiable author with public social media profile?

  • Does the publication have a history of political bias or allegiance?

  • Is the web “footprint” verifiable (domain, hosting company, hosting location, advertising code, security certificate).

• Writeprint – Content writing style:

  • Is the article written in a balanced, non-sentimental way?

  • Is the quality of the text consistent with the topic of the article?

  • Does it contain actual and useful information?

  • Does it mention the information sources it uses (including video/photo)?

  • Were the images used before on the web in a different context?

  • Does it contain strong emotions towards the subject or use of clickbait techniques?

  • Does it use fraudulent images, out-of-context videos, claims that cannot be checked?

  • Does it use anonymous or unnamed sources?

• The information origin and context:

  • Can the information be traced back to the original (Patient Zero) information?

  • How trusted is the origin of information and other content sources referenced?

  • What are other publishers and articles writing about the same topic?

In fact, these verifications which are performed automatically by software algorithms relying on text analytics, graph analytics, and statistical analysis provide a good baseline for the standardization and automation of fake news detection and analysis. Also, the impact of fake news stories can also be measured using standard metrics:

• 1. Facebook or Twitter engagements, such as likes, shares, comments, retweets. The higher these metrics are, the more people were reached by the content (have read at least the title) and have interacted with it, distributing it further in their networks by linking it, retweeting it, or commenting on it.

• 2. Content engagement analytics, typically available only to the content owners, showing the number of visits on their website or views on social media.

• 3. References and citations, either direct hyperlinks to the article’s URL from other websites or just mentions (“according to...”).

The advancements in Artificial Intelligence algorithms and processing power allows software programs to automatically extract key information from content with no human supervision: language detection, automatic summarization, named entity extractions, sentiment analysis, hyperlinks or references, clickbait detection, content classification, emotion extraction, semantic similarity comparison. All these metrics can be used to understand content in a similar way as a human does, with the goal of assessing its trustworthiness.

6 Fake News: “Red Flags”

When creating fake news, both the content and distribution channels are important. The content needs to be appropriate for the end goal: fringe websites will use clickbait techniques and rely on social media, state actors will create high-quality content and try to send the message into the mainstream, conspiracy theorists will use message boards and fora with “source hacking” techniques.

The most typical techniques or “red flags” for creating and distributing disinformation are:

• Using clickbait writing style, especially for the article title, with the main purpose of attracting attention and encouraging visitors to click on the hyperlink or distribute the content. Certain punctuation marks, heavy use of capitalized letters, words and expressions that trigger strong emotions, and the lack of actual information in the title (names, locations), the different elements of information presented in the body of the article in comparison with the title, are the markings of a “clickbait” content. The generation of content by every day users of social media platforms (UGC – user generated content), exhibiting large variation in information quality [3], poses additional and increasing challenges.

• Republishing old stories in order to enforce the Anonymous or unverified sources. As the Internet has democratized content production, information published by sources that have no contact details, no ownership information or anonymous authors, should be regarded as untrustworthy. Even if the publishers are anonymous, trust can be built up in time, if the information published is constantly verifiable and trustworthy.

• Lack of factual information in the articles. Opinion pieces, conspiracy theories, or low-quality journalism tend to exhibit a lack of information context – people, organizations, geographical, and time references – elements that typically constitute the factual information backbone of any news story.

• Relying on unnamed sources, such as unnamed officials, anonymous tipsters, or political insiders, is a technique used by established media organizations in order to either get ahead of the competition with breaking stories or to create propaganda stories. Although this information sourcing is accepted by most media organizations, it can be considered untrustworthy. These types of stories tend not to get verified afterward.

• Lack of sources or using unreliable sources. Since information is often created by news agencies or local newspapers, witnessing events first hand, the vast majority of other publication types typically rely on sources for their stories. The lack of sources referenced in the articles or referencing unreliable source constitutes a “red flag” leading to possible untrustworthiness. The generation viewpoints of certain online “echo chamber”-like groups are a known and potentially successful fake news technique, since the huge amount of news content produced every day ensures that the public cannot remember old stories and be assumed to react to the same story in the same way when confronted with republished news.

• Source hacking, a complex fake news creation technique specific to forums and message boards where anonymous users disguised as an insider, firsthand witness, or whistleblower information would publish fake information in the hope that journalists on the look for stories will pick up the information and publish it in the mainstream media. News organizations with established source verification and quality standards tend to keep away from such anonymous online information.

• Use of manipulated images or wrongly attributed videos, either as part of written news articles or standalone, which can be difficult to debunk, making it another successful fake news technique. The “red flags” for such content can only be obtained with specialized software and rarely verified using journalistic or critical reading techniques.

• Different viewpoints also make up an important verification process for any information. If other publishers are writing about the same topic, if there are different viewpoints on a specific matter, this is a sign that the story is not fake. Even if the stance on the topic differs from publisher to publisher, even if there is no consensus, witnessing multiple actors engaging with a story is a valuable verification starting point, this can be assumed to be the actual case.

The content red flags and engagement models can differ depending on the type of fake news. To summarize the 3 most popular types of written fake news content:

• Clickbait, which is usually created by non-established news websites and relies mostly on the writing style to obtain maximum audience reach, typically on social media platforms. This type of disinformation used either for political or commercial gains does not reference sources or does it only partially or in a deceptive way. The content does not contain sufficient factual information, rarely has a named and identifiable author, and the main focus is actually on the title written in such a way that it should motivate the reader to click on it (hence click-bait) or redistribute it without reading the full article. The content can also be overly negative or positive. Rarely cited or referenced outside the content producer’s own network, the aim of this content type is to “viralize” (becoming viral), to obtain a large number of views, likes, shares or retweets on social media platforms.

• Conspiracy, websites that publish mostly false information on different topics such as history, politics, religion, or health. Conspiracy websites tend to have a varied degree of popularity and present information that cannot be proved scientifically. The writing style can vary from clickbait-type content to high-quality content, resembling scientific papers. The Wikipedia page on conspiracy theories [69] lists 12 categories of conspiracy theories. Recently, message boards such as 4chan or 8chan have been successful at collaboratively creating and distributing conspiracy theories in the online space, one of the most famous users involved in these activities being QAnon [15].

• Propaganda, sponsored by various political organizations or even state-actors. Propaganda stories may span a large number of articles and are typically information-rich, displaying a high-quality writing style, and referencing a known author in order to resemble high-quality journalism and not be labeled as clickbait of fake news. However, even if propaganda articles make use of multiple sources, they tend to mix trusted and verifiable information with untrusted, usually unnamed sources or other websites that are partisan.

The following examples detail the various types of fake news, from misinformation to disinformation, highlighting the various techniques of creating and distributing content, and the impact they had on the economy, society, and politics.

7 Examples

7.1 The Volkswagen Emissions Scandal Spills over to BMW

Even before the “fake news phenomena” made international headlines due to the 2016 US presidential elections, scandal, disinformation, misinformation, and unverified claims were plaguing the news ecosystem with considerable impact. One prominent example is how during the Volkswagen emissions scandal in the autumn of 2015, a German magazine managed to significantly impact the stock value of auto giant BMW after publishing an unverified claim that went viral.

On the September 18, 2015, the US Environmental Protection Agency (EPA) issued a Notice of Violation for Volkswagen AG, Audi AG, and Volkswagen Group of America, Inc., accusing the company group of installing “defeat” devices in certain car models that cheated on harmful emissions measurements. In fact, the EPA was accusing the VW Group of “rigging” their car emissions measuring devices by showing values that were lower than the real ones in order to conform to the strict US environmental regulations. The Notice of Violation was published on the EPA website [26] and was quickly picked up by the media the following day.

On September 19th, there were already several thousand online articles on this topic, according to Google Search, making it one of the biggest news stories of that week. One day later, VW CEO Martin Winterkorn confirmed the data published by the US EPA to be true. By September 21st, the VW stocks had plummeted by 20% [16] and a Wikipedia page on the “Volkswagen emissions scandal” [70] had been created.

While the media was busy trying to understand the magnitude of Volkswagen’s emissions rigging operation and also trying to assess the impact of this scandal on the auto industry worldwide, the German magazine AUTO BILD published an article claiming the rival BMW might have the same problem, stating that the BMW X3 model produced more than 11 times the European limit when tested by the International Council on Clean Transportation (ICCT). Published on September 24th under the title “AUTO BILD exclusive: BMW diesel exhaust emissions exceed limits significantly” (original title: AUTO BILD exklusiv: Auch BMW-Diesel überschreitetAbgas-Grenzwertedeutlich)¹³.

The article by AUTO BILD was picked up instantly and without proper verification by many publications and online blogs around the world, resulting in an immediate drop of BMW stock by nearly 10% [17]. BMW management denied any wrongdoing the same day and AUTO BILD modified the original article publishing a correctional statement: “No indications of manipulation from BMW” (“KeinIndizfür Manipulation bei BMW”) (Auto [10]).

The mistake by AUTO BILD and its consequences can be regarded as a “textbook” example of disinformation – a type of fake news produced without intent. The effects of this disinformation by AUTO BILD were the same as for any other fake news, even if BMW was quick to deny the claim and the German auto magazine published a clarification only shortly afterwards. The information was quickly picked up by other media outlets and websites with no proper verification, mainly because AUTO BILD was regarded as a trustworthy news source, it spread rapidly in the online space, had significant impact (BMW stock crashing) and only a very small segment of those who picked up the information also published the correction afterwards.

7.2 Mainstream Media Is no Stranger to Fake News

If misinformation is produced when the editorial process does not enforce certain rules that have to do with content quality and source verification, disinformation is fake news created on purpose, whether by hyper-partisan blogs that use clickbait-techniques to spread hate speech, or false information about political adversaries, or by mainstream media actors using complex propaganda techniques in an effort to shape a wider audience’s views on certain national or global topics.

One of the most frequent disinformation techniques employed by the media, especially when dealing with political topics, is using anonymous sources of information, referred to as “sources close to the matter,” “experts that wish to remain unnamed,” “undisclosed official sources,” “confidants,” or “people familiar with the case.”

A relevant example of such disinformation carried out by the mainstream media is the alleged US arrest warrant for Wikileaks founder Julian Assange. On April 20, 2017, several US news publications reported the US Government was preparing charges for the arrest of Wikileaks founder Julian Assange. However, this information did not come from (nor was confirmed by) a named US official or institution. Leveraging statements from CIA Director Mike Pompeo and Attorney General Jeff Sessions on the Wikileaks investigation (dating back from 2010), the claim that the US was preparing to charge Assange and Wikileaks was attributed to “US officials familiar with the matter” and “people familiar with the case.” The claim was never confirmed and remains so until today.

The information that the “US prepares charges to seek arrest of Wikileaks’ Julian Assange” was first published by CNN [18] and then by The Washington Post [67] on April 20, 2017, just 13 min apart. The information was subsequently picked up by several other prominent news organizations, such as Newsweek, Deutsche Welle, USA Today, The Verge, The Blaze, Russia Today, and the BBC.

In fact, the information source cited by both CNN and the Washington Post was “US officials familiar with the matter” (CNN) and “according to people familiar with the case” (Washington Post). More than a year later, the US Government had not put forward any formal charges against Julian Assange, proving that either the sources were misinformed or were using disinformation. It is also possible that the publishers were either deceived by their unnamed sources or were just too quick to publish information that was impossible to verify.

It is commonly known that the use of unnamed sources for news stories can be accepted under certain conditions and circumstances. Frequently journalists have a choice between relying on unnamed sources and not publishing a story at all. Relying on unnamed sources that have produced reliable information in the past can be virtually risk-free. However, dealing with information that is not precise and lacks specifics can be an indication that the source is unreliable or even outright wrong. In such cases, journalistic quality standards should take precedence over being first and “breaking a story” before the competition, risking to produce fake news and lose the audience’s trust.

Another example of mainstream media-created “fake news” is that of ABC News claiming that presidential candidate Donald Trump instructed retired Lt. Gen. Michael Flynn to “contact the Russians,” an action that would be considered illegal in the USA. The story created insecurity in the financial markets and caused the Dow Jones to drop by 350 points.

On December 1, 2017, ABC News posted a message on Twitter regarding a journalistic investigation by senior journalist and anchor Brian Ross, gathering 25,000 retweets in the first hour:

“JUST IN: @BrianRoss on @ABC News Special Report: Michael Flynn promise full cooperation to the Mueller team and is prepared to testify that as a candidate, Donald Trump directed him to make contact with the Russians.”

This information, as part of the ongoing Russian collusion scandal, was aimed to confirm that candidate Donald Trump had colluded with Russia during the US Presidential Elections in 2016. After Michael Flynn was arrested and pleaded guilty to lying to the FBI on Friday, Ross reported that Flynn planned to tell authorities that President Trump had personally directed him to establish contact with Russia while he was running for president. This explosive claim, which suggested a new dimension to Robert Mueller’s investigation into Russian election interference, was picked up by many other journalists, and even caused a significant, temporary dip in the stock market, according to NYmag.com [46].

Brian Ross was suspended by ABC over the false Trump report the next day and apologized for the “serious error we made yesterday.” Journalist Brian Ross claimed that the mistake came from an unnamed source (referred to as “the confidant”) that “had actually told him that Trump had made the comments in question when he was the president-elect, not a candidate,” but after the original information had made it into his article.

The original ABC Twitter message containing the mistake was distributed 25,000 times before it was deleted and the referenced article [2] topped 100,000 Facebook interactions, as measured with www.sharedcount.com on December 11, 2017. More than 8500 news websites around the world picked up and republished the information, according to Google News search. Not all of the websites that picked up the story also published the ABC correction or the fact that author Brian Ross was suspended for his mistake. The quantifiable impact of this misinformation or disinformation was a significant drop of the Dow Jones by 350 points, an impact similar to the one created by AUTO BILD’s mistake regarding BMW’s emissions values.

7.3 Conflict Zones, the Playground for Propaganda

A very rigid approach on information verification could have a binary classification viewpoint: true or false. However, when dealing with information originating from conflict zones, as for example the war in Syria, the notion of “unverifiable information” comes into play, due to the multiple actors trying to control the information and the fact that on-the-ground journalistic information verification is nearly impossible. The Syrian conflict has been marked by constant propaganda and disinformation campaigns since its beginning in 2011 with all involved belligerents, whether Bashar al Assad and his allies, the Western Coalition, or the Islamic State.

One example from October 2015 is the news that a bombing by Russian warplanes of a hospital in the city of Sarmin, in the Idlib Governorate, Syria, resulted in the death of eight civilians. The article was published on the SOHR – Syrian Observatory for Human Rights – website www.syriahr.com (edited by Rami Abdurrahman, a Syrian national living in the UK) on October 2015 [57], but is not available anymore. On October 21, 2015, the information from the SOHR expanded into a full story for the AFP and subsequently was picked up by most major news agencies including DW, Radio Free Europe, Mail Online [21], NDTV [44], and Sky News, citing the SOHR website administrator, Rami Abdurrahman. On October 22, 2015, the information was then confirmed by the Syrian-Medical Society (Syrian American Medical Society or SAMS) by posting a picture of the alleged bombed hospital on Twitter. This was followed by a statement on their website [50] claiming the medical facility was a SAMS polyclinic in Sarmin, Idlib and that it was targeted by Russian airstrikes using air-to-surface missiles. The statement furthermore provided exact figures on the number of casualties, the SAMS staff members killed in the attack and four pictures of the aftermath that are now missing from their website.

The state-funded news agency Russia Today published a statement by the spokesperson of the Russian Foreign Ministry on the matter the next day [48]. Maria Zakharova denied any Russian bombing of hospitals in Syria, claiming the report showed tremendous bias towards Russia’s military efforts in Syria:

There are so-called mass media reports which allege that Russian aircraft bombed a field hospital in the Idlib Governorate in northwestern Syria and reportedly killed 13 people. I cannot say that these reports are written by journalists but their ingenuity delights.

RT.com then followed with an update on the story on October 29 [49], publishing a statement from Dominik Stillhart, director of operations at the International Committee of the Red Cross, (which has people on the ground in Syria), saying that he was unaware of any such incidents: “We’ve seen these reports as well, but in the absence of any firsthand information coming from our teams on the ground, I can neither confirm, nor deny these allegations.”

The conclusion of Dominik Stillhart summarizes very well the way stories from conflict zones should be approached. As the source of this story is a statement from a Syrian activist (Rami Abdurrahman), followed by more information from a US-funded medical association (SAMS) featuring images that could not be verified, with no real follow-up on the story in order to find more evidence or witnesses and on-the-ground footage from a reliable independent source, it is actually impossible to say whether this story is true or false.

The same goes for another, more recent story regarding the Syrian conflict: the bombing of Syrian army positions by the US Army on May 24, 2018. The information was published by several media outlets, only to be disputed shortly afterwards. The source of information was traced back to the Reuters World Twitter account (twitter.com/ReutersWorld), citing the Syrian Arab News Agency (SANA):

U.S.-led coalition hits Syrian army positions: Hezbollah media unit https://reut.rs/2IKBane.

The tweet linked to the story that was updated quickly after the post on Twitter: “Syrian state media says U.S. hit army posts, U.S. denies” [47] tracing the information source to Syrian state media agency SANA that was citing “a military media unit run by Lebanon’s Hezbollah.” The information was denied by US military official Captain Bill Urban, a spokesman for the US Central Command as well as by Russian military sources cited by RT.com.

The original post of Reuters was published on the SANA website [51] at 2018-05-23 23:40:42 and was deleted shortly thereafter. But not before being referenced by a number of Arab-language publications, including the Syrian Observatory for Human Rights and international news agencies. The SANA article, authored by “mohamad” only contained a short sentence (retrieved through search engine cache and translated from Arabic):

“Military source: Some of our military positions between Albuqmal and the intimacy of the dawn of the day to the aggression launched by the US coalition aircraft in conjunction with the rallies of the terrorists of the organization advocated and limited damage to material”

One hour later, the same author (mohamad) published a more detailed piece on the same story [52] (translated from Arabic): “Within the framework of its support for the “daaish terrorists. The US coalition attacks on some of our military Prevlaka (rural) Deir Al-Zour.”The article used the same anonymous source as the previous article and mentioned that “The aggression after less than 24 hours to foil the Syrian Arab army forces and the generic attacked terrorists from “Daaish” on a number of points in the apparent fields Prevlaka (rural) Deir Al-Zour and the elimination of more than 10 of them some of them foreign nationalities and dozens of injured and the destruction of one of their vehicles.” It also specified that there were no injuries on the Syrian side.

The information was quickly picked up by Arab-language publications:

• Al Majd [7]. Citing SANA, claiming the loss of 25 civilian lives. The article has been deleted since.

• Asharq Al Awsat [1]. Using the information from SANA.

• Alsumaria [8]. Citing SANA, but mentioning the US did not confirm the attack. In addition, it mentioned that Syrian military sources claimed the attack targeted two sites near the T2 Atomic Energy facility, which is located near the border with Iraq, about 100 km west of the Euphrates River.

• Al Bawaba [4]. Identical to the AlSumaria article.

• Sky News Arabia [56].The first Arab-language publication to mention that the source was actually Hezbollah.

• Aljazeera [6]. Mentioning that the Pentagon did not confirm the attack and attributing the information to the Syrian Government (SANA).

English-language publications that picked up the story and then published the Pentagon’s denial were: The Associated Press, North Jefferson News, The Seattle Times, The Washington Post, New York Post, Russia Today, The Times of Israel.

This story can be considered a successful source-hacking attempt orchestrated by Hezbollah. Publishing unverifiable information on the Syrian state news agency website SANA was carried out in order to be used by other Arab-language news websites and hopefully also by English mainstream news agencies, such as Reuters. Again, the unnamed Hezbollah source, the impossibility to verify the claim, and the risk of being just a daily piece of wartime propaganda did not prevent this story from reaching mainstream media in the English-speaking world although it was denied afterward by US and Russian military sources. It also provides an example of cross-language and multilingual efforts, making any verification and debunking process even more difficult.

Another example of how propaganda works in the Syrian conflict zone was debunked by the Atlantic Council’s Digital Forensics Lab¹⁴ in September 2017: “How Kremlin-backed and fringe media spread a false story claiming the U.S.-led Coalition evacuated ISIS from the front lines.” The investigation focused on an article by Russian News Agency (Sputnik) trying to trace the information back to its original source [42].

Confirming the pattern, Sputnik’s information source is an anonymous “military and diplomatic” source, while another website – the Syrian-based Deirezzor24.net [25] – cites “A D24 correspondent.” Although the information did not reach mainstream media, it was picked up by conspiracy media outlets such as thefreethoughtproject.com, Al-Masdar News, The Fringe News, or The Duran.

One of the obvious patterns of disinformation in the Syrian conflict is that from the main belligerents – the Syrian government, the Russian military, and the US-led coalition – there is rarely any consensus on information among the three actors. When information is published by the Syrian Government, for example, it is quickly denied by the USA. When it is published by the USA and mainstream media, it is labeled as propaganda by the Russian officials and so on. In the end, the control of information on the Syrian conflict is just another facet of the way modern war – an information war – is conducted, with the public having great difficulty assessing the trustworthiness of information regarding this civil war which has been raging on since 2011.

7.4 Clickbait-Type Fake News for Political Gains

The term “fake news” came into global attention shortly after the US presidential elections in 2016 when the victory of Donald Trump was marred with accusations of his supporters using disinformation and clickbait techniques in the online space and more specifically on Facebook to target his opponent, Hillary Clinton, and the Democratic Party. Some statements even went so far as to claim that fake news might have gotten Donald Trump elected, as stated in an article by The Guardian [33] as “the most obvious way in which Facebook enabled a Trump victory has been its inability (or refusal) to address the problem of hoax or fake news.”

According to TechCrunch [61], the issue of spreading misinformation on the social media platform was real and was admitted by Adam Mosseri, VP of product management at Facebook, who conceded that “the company does need to do more to tackle this problem.” Later, Facebook founder Mark Zuckerberg denied that this phenomenon had an influence on getting Trump elected: “To think it influenced the election in any way is a pretty crazy idea” (statement made during the Techonomy conference in Half Moon Bay, Calif, cited by USA Today [66]). The culprits for this “fake news surge”, as it was called by the AI-startup Zetta Cloud [62], were anonymous supporters from all over the globe engaging in a sustained campaign to publish and distribute fake news on social media, with the aim of lowering the US public’s trust in the Democratic Party and candidate Hillary Clinton.

The claim that this fake news campaign was solely responsible for Trump’s victory was later corrected by a study pointing out that wide reach does not equal strong impact [24]. But the numerous sites that are still spreading fake news with strong “clickbait” recipes represent the bulk of “fake news” sites known to the general public. Some examples of this type of hyper-partisan fake news show obvious patterns of how this type of disinformation can have such a wide reach in online communities.

The post “Damning Hillary Clinton Footage Leaks – The Truth is Out” [64] published on May 14, 2017, by TheTruthMonitor.com ¹⁵ indicates another right-wing fake news story that is set out to “viralize” quickly and be instrumented as political propaganda. The title writing style aimed at generating strong emotions and a desire to click the headline (thus the term clickbait) containing misleading information is something that is commonly used when creating fake news.

The article, having as author “Adam Lott” (with no contact details or social media profile), trashes Hillary Clinton and her supporters in the context of Donald Trump firing FBI Director Comey while explaining the previous firing of state attorney Preet Bharara by using the precedent of Bill Clinton firing all of Bush’s state attorneys back in 1993. Even though the actual content of the article and the sources it references (a tweet from CNN supporting state attorney Preet Bhahara and a tweet from political commentator Dinesh D’Souza [65] do not present any “damning information” or any “footage leaks “regarding Hillary Clinton, this didn’t stop the article from going viral, gaining more than 10,000 Facebook engagements.

Another example is the targeting of other Democrats, such as America’s soon-to-be first female Muslim legislator, Ilhan Omar. In August 2016, the website “World Net Daily” [71] published a story accusing Ilhan Omar of being married to two men at the same time, including one who may be her brother. The story, published during Minnesota’s August 9, 2016, Democratic primary, won by 33-year-old Somali refugee Omar, originated from the blog of lawyer Scott Johnson¹⁶ and resulted in what the newspaper Minnesota Post [43] called a “five-day brush fire” causing serious issues for the candidate and resulting in a clarification: Ilhan Omar was married to only one man, who had changed name from Ahmed Hirsi to Ahmed Aden, hence the confusion.

Although this story was clarified in 2016, it reappeared 1 year later, on July 20, 2017. Several right-wing blogs supporting Donald Trump, such as dailyinfo.co, usanewspolitics.com, conservativefighters.com [19], TeaParty.org, and angrypatriotmovement.com, published a post with the exact same title: “JUST IN: Muslim Congresswoman Caught In SICK Crime. Should She Go To Prison?”

The posts, which together gathered close to 40,000 Facebook interactions, claim that Ilhan Omar married her own brother in a scheme to fool US immigration officials and supported refugee fraud in her state. The content of these posts picks up the year-old information from World Net Daily without the debunking and the clarifications, and “spices it up” with the information that Omar might go to jail. It employs the successful technique of taking old information that was debunked in the meantime and using it for the same purpose, knowing that people rarely read or remember the corrections of fake stories. To accelerate the distribution speed, the clickbait title makes sure to be highly aggressive, suggesting that the scandal just happened and not mentioning the congresswoman’s name.

Equally, on the other side of the US political spectrum, there is no reluctance to use clickbait titles to ensure a wide audience reach by mixing half-truths in order to obtain political gains. The liberal LA-based blog yournewswire.com has already gathered more than 106,000 Facebook interactions for an article published in May 23, 2018: “20 Priests Connected To Vatican Pedophile Ring Killed In Plane Crash” [72].

After a tragic plane crash in Cuba on May 19th, claiming the lives of 20 priests [23], Daily Mail authors link this information with a child sex abuse scandal from Chile involving the Catholic Church [22], stating that the priests perished in Cuba were pedophiles. Besides a “clickbait”-style title, the article uses a technique of merging disconnected information and ineptly “connects the dots” in order to create a fake news story that gathered significant traction on Facebook. YourNewsWire, a known fake news website, is published by Sean Adl-Tabatabai and Sinclair Treadway and has made the headlines in mainstream media for being an “alt-left agitator website”[35].

7.5 When Whole Countries Fall Victim to Fake News

Moving away from the English-language online space, fake news phenomena can become a nationwide critical problem in areas where digital literacy is low among the population. One relevant example is the Republic of Indonesia, with a diverse population of 261 million and a fake news problem that affects every part of its society.

From the smear campaign against Indonesian president Joko “Jokowi” Widodo in 2015 to Jakarta’s Christian and ethnically Chinese governor Basuki Tjahaja Purnama, popularly known as Ahok (who was targeted by hardline Muslims, who successfully pressured the authorities to put him on trial for blasphemy in 2016), fake news is used in an organized and almost weaponized manner by political groups in Indonesia, resulting in mass protests, political crisis, and religious and ethnic hate.

In February 2018, the Indonesian government established a National Cyber and Encryption Agency (BSSN) with the goal of fighting the fake news phenomenon that has engulfed the country’s political scene. A plan to establish the BSSN was put forth 4 years when president Joko Widodo took office, but the agency only started work in January 2018 after Major General Djoko Setiadi was installed as its leader [59]. In fact, current president Joko Widodo was the target of a smear campaign during the presidential elections of 2014 when a social media campaign was circulating rumors that the popular Jakarta governor was an ethnic Chinese and Christian – a sensitive issue in a Muslim-majority country with a low digital literacy level. Called “black campaigns” in Indonesia, these were aimed at hurting Widodo’s electoral score in favor of more radical Muslim candidates. Pictures were circulated of a fake marriage certificate claiming that Widodo is of Chinese descent and originally a Christian. The messages were spread over Blackberry message groups, WhatsApp, Facebook, and Twitter.

Featuring among the top five biggest users of Facebook and Twitter globally, Indonesia has faced regular and very effective fake news campaigns: from memes, like the one targeting social-media activist UlinYusron, to edited videos, like the one that got Jakarta’s governor put on trial for blasphemy, to online-organized bounty hunts against more than 100 individuals who saw their personal information released online.

The Indonesian authorities are expecting a similar surge of fake news for the 2019 elections. Recently, according to the Jakarta Globe (Jakarta [36]), the group orchestrating most of these fake news campaigns, called the Muslim Cyber Army, was exposed and its members arrested. The group was accused of running an online army of Twitter bots, messaging groups, and fake accounts with the purpose of spreading fake news and smear campaigns against certain political actors while also fueling a state of paranoia and tension against the Christian and Chinese minorities.

8 Conclusion and Outlook

Fake news is an age-old phenomenon, from the disinformation campaigns in ancient empires, to the recent social-media campaigns aimed to influence election results. Along this timeline, there seems to be a balance between the skills and tools required for people to protect themselves from disinformation and the methods and distribution channels employed by the creators of fake news. When the distribution channel for fake news was the word-of-mouth, people would rely on their common sense and on the messenger to assess the trustworthiness of the information. Nowadays, digital literacy and the possibility to verify information in real-time across multiple sources (and some common sense) are the basic tools.

Even if the media agree on content creation models that enforce transparency and result in high-quality trusted content, even if online distribution channels automatically filter out or flag untrusted content using up-to-date algorithmic approaches, the Internet remains a place where anyone can create and distribute content easily. The importance of the human factor in the dissemination of fake news and the responsibility in the sharing of information cannot be stressed enough. Ultimately, the burden to judge the content rests with the audience, the content-consumers.

Digital literacy education programs are a must in the age of the Internet. A dedicated focus, both financial and academic, into the creation of software and tools to help content creators, distributors, and consumers to easily identify and filter-out untrustworthy content, is the technology backbone that will result in an applicable and scalable method to detoxify online information. Ethical issues such as possible censure and freedom of speech need to be taken into account from the start.

When it comes to being aware of what information to trust, the burden on the audience can be immense. Information quality and perceived information quality play a fundamental role, in particular on social media platforms, where the sharing of information is one of the key elements. The inclination to share rapidly and in an unreflected manner is only occasionally balanced by the fear of loss of reputation.

Continuous education about the techniques and patterns typically employed by producers of fake news, becoming immune to clickbait titles and resisting the tendency to immediately interact with and share content that has not been verified or that originates from untrustworthy sources, making a habit of cross-checking information from other sources, and using news database services and image verification tools – these and related measures and activities cause a heavy load and require knowledge and skills in order to stay “fake news free.” In sectors and societies where social media platforms and the Internet are still a new thing, this task can become nearly impossible.

Looking further into the AI future, conversational bots that can be easily mistaken for human users, deep fakes created with advanced image and video-processing technologies, the emergence of content that is created automatically – “robot journalism” – these new advancements will add even more complexity to the online information creation and dissemination landscape.

The solution needs to be one that will work in the long-term and is most likely to be a hybrid one: establishing digital literacy education as a must, from an early age, across as many educational systems as possible, combined with a continuous effort to create and deploy AI-based software that automate most of the tasks required to identify fake news and support consumers in their struggle.

The two approaches have little chances to be successful by themselves individually. Digital literacy educational programs are now mostly operated by non-governmental organizations; it will still take some years before they are adopted, adapted, and improved by governments and made part of the regular educational curricula. And even when they are, they will still need to keep up with the ever-new ways of creating and distributing “fake news.” On the other side, AI algorithms, even in their most advanced form, will probably never be as good as humans, when it comes to analyzing complex culture- and language-specific disinformation campaigns. But AI’s goal is not to replace human assessment, but rather provide a scalable method of addressing the most obvious forms of disinformation and to be able to quickly extract insights for humans to review, shortening investigation times. Time and scale are of essence and can be addressed by AI technologies. The two approaches, taken together, have the best chances of curbing the fake news surge in the online space, given that enough focus is given from society’s leading actors: governments, academia, and technology entrepreneurs. These efforts need to be accompanied and supported by further research into the detection of fake news, its dissemination, the role of information quality, and perceived information quality in such processes and how such insights could readily be incorporated into social media platforms to support users in their struggle. At the same time, commercial entities need to embrace such advances and provide tools and plug-ins allowing to put these results into practice.