Abstract
Throughout the COVID-19 pandemic, people have grown more reliant on social media for obtaining news, information, and entertainment. However, the information environment has become a breeding ground for disinformation tactics. Formal recommendations from medical experts are becoming muffled by the avalanche of toxic content and social media echo chambers are being created in hopes that users only consume stories that support certain beliefs. Despite the advantages of utilizing online social networks (OSNs), a consensus is emerging suggesting the presence of an ever-growing population of malicious actors who utilize these networks to spread misinformation and harm others. These actors are using advanced techniques and are engaging on multiple platforms to propagate their disinformation campaigns. As such, researchers have had to evolve their methods to detect disinformation. In this chapter, we present novel multimethod socio-computational approaches to analyze disinformation content and actors on OSNs during the initial months after COVID-19 was made public. These techniques are presented as case studies in narrative analysis of COVID-19 misinformation themes on blogs, identifying anti-lockdown protestor coordination through connective action on Twitter, analysis of hate speech and divisive discourse on YouTube through toxicity analysis, and modeling of misinformation contagion using an epidemiological approach. We end the chapter by presenting a COVID-19 misinformation tracker tool developed in collaboration with the Arkansas Office of the Attorney General. Our results offer policymakers valuable data to make informed decisions about the information environment and derive appropriate and timely countermeasures to combat insidious forms of cyber threats. Our efforts demonstrate that when researchers coordinate with policymakers it can make a difference, especially when that coordination remains an ongoing process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Blackbird.AI provides Disinformation Defense and Response to national security and enterprise customers. https://www.blackbird.ai/history-of-misinformation/.
- 2.
EUvsDisinfo is the flagship project of the European External Action Service’s East StratCom Task Force established in 2015 to better forecast, address, and respond to the Russian Federation’s ongoing disinformation campaigns affecting the European Union. https://euvsdisinfo.eu/.
- 3.
BitChute is a video hosting service claiming to put creators first and provide them with a service that they can use to flourish and express their ideas freely. https://www.bitchute.com/.
- 4.
Colors denote toxicity scores from lowest, 0.5 (blue) to highest, 1 (red). Since toxicity scores are based on a probability score of 0 to 1, we focused our analysis on toxic comments that are 0.5 or greater in order to gain a deeper understanding of high toxic content.
References
Agarwal, N., Al-khateeb, S., Galeano, R., & Goolsby, R. (2017). Examining the use of botnets and their evolution in propaganda dissemination. NATO Journal for Defense Strategic Communications, 2, 87–112.
Agarwal, N., & Bandeli, K. K. (2020). Examining strategic integration of social media platforms in disinformation campaign coordination. NATO Journal for Defense Strategic Communications, 4(1), 173, 2018.
Agarwal, N., Lim, M., & Wigand, R.T. (2012). Raising and rising voices in social media: A novel methodological approach in studying cyber-collective movements. Business & Information Systems Engineering (BISE) Special Issue on IS and Culture, Dorothy Leidner (Ed.), WIRTSCHAFTSINFORMATIK. 2012. https://doi.org/10.1007/s12599-012-0210-z
Alassad, M., Hussain, N., & Agarwal, N. (2020). Systems thinking and modeling in social networks: A case study of coronavirus conspiracy theories challenges in Twitter network. Presented at the SBP-BRiMS 2020, October 2020.
Al-khateeb, S., & Agarwal, N. (2019). Deviance in social media and social cyber forensics: Uncovering hidden relations using open source information (OSINF). Springer Briefs in Cybersecurity. Springer, 2019. ISBN: 978-3-030-13689-5.
Al-khateeb, S., Conlan, K. J., S., Nitin, A., Ibrahim, B., & Frank, B. (2016). Exploring Deviant Hacker Networks (DHM) on social media platforms. Journal of Digital Forensics, Security and Law, 11. https://doi.org/10.15394/jdfsl.2016.1375.
Allyn, B. (2020, June 16). Study exposes Russia disinformation campaign that operated in the shadows for 6 years. NPR. https://www.npr.org/2020/06/16/878169027/study-exposes-russia-disinformation-campaign-that-operated-in-the-shadows-for-6-. Last accessed 28 Feb 2021.
Ananyev, M., & Sobolev, A. (2017, April). Fantastic beasts and whether they matter: Do internet ‘trolls’ influence political conversations in Russia? Presented at Midwest Political Science Association, Chicago, IL.
Anderson, J., & Moody, K. (2020). Stigmas and filter bubbles: How the Russian misinformation campaign in the 2016 US presidential campaign links to stigma communication. Iowa Journal of Communication, 52(2)‚ 31–47.
Badawy, A., Ferrara, E., & Lerman, K. (2018, August). Analyzing the digital traces of political manipulation: The 2016 Russian interference twitter campaign. In 2018 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM) (pp. 258–265). IEEE.
Banerjee, D., & Rao, T. S. (2020). Psychology of misinformation and the media: Insights from the COVID-19 pandemic. Indian Journal of Social Psychiatry, 36(5), 131.
Bao, H., Cao, B., Xiong, Y., & Tang, W. (2020). Digital media’s role in the COVID-19 pandemic. Journal of Medical Internet Research mHealth and uHealth, 8(9), e20156.
Barthel, M., & Mitchell, A. (2017, May 10). Americans’ attitudes about the news media deeply divided along partisan lines. Pew Research Center’s Journalism Project. https://www.journalism.org/2017/05/10/americans-attitudes-about-the-news-media-deeply-divided-along-partisan-lines/. Accessed 13 Nov. 2019.
Bettencourt, L. M., Cintron-Arias, A., Kaiser, D. I., & Castillo-Chavez, C. (2006). The power of a good idea: Quantitative modeling of the spread of ideas from epidemiological models. Physica a: Statistical Mechanics and Its Applications, 364, 513–536.
Bond, S. (2021, February 8). Facebook widens ban on COVID-19 vaccine misinformation in push to boost confidence. NPR. https://www.npr.org/2021/02/08/965390755/facebook-widens-ban-on-covid-19-vaccine-misinformation-in-push-to-boost-confiden. Last accessed 28 Feb 2021.
Burkhardt, J. M. (2017). History of fake news. Library Technology Reports, 53(8), 5–9.
Burton, A., & Koehorst, D. (2020). Research note: The spread of political misinformation on online subcultural platform. Harvard Kennedy School, Misinformation Review. https://misinforeview.hks.harvard.edu/article/research-note-the-spread-of-political-misinformation-on-online-subcultural-platforms/. Last accessed 28 Feb 2021.
Calabresi, M. (2017). Inside Russia’s social media war on America. https://time.com/4783932/inside-russia-social-media-war-america/. Last accessed 23 Feb 2021.
Center for Strategic & International Studies. (2020 September 23). Countering Russian disinformation. https://www.csis.org/blogs/post-soviet-post/countering-russian-disinformation. Last accessed 28 Feb 2021.
Cha, M., Gao, W., & Li, C. (2020). Detecting fake news in social media: An Asia-Pacific perspective. Communications of the Association for Computing Machinery, 63(4), 68–71. https://doi.org/10.1145/3378422
Clark, M. (2020 September 19). Misinformation poses ‘severe test’ in Asia Pacific. Food Safety News. https://www.foodsafetynews.com/2020/09/misinformation-poses-severe-test-in-asia-pacific/. Last accessed 28 Feb 2021.
Chou, W. Y., Gaysynsky, A., & Cappella, J. N. (2020). Where we go from here: Health misinformation on social media.
Commission on Security and Cooperation in Europe. (2017, September 14). The scourge of Russian disinformation. https://www.csce.gov/international-impact/events/scourge-russian-disinformation. Last accessed 28 Feb 2021.
Commonsense.org. (2017). Fake news: Historical timeline. https://www.commonsense.org/education/sites/default/files/tlr-asset/newsmedialit_fakenewstimeline_8.5x11.pdf. Last accessed 28 Feb 2021.
Compartmental models in epidemiology. (2021, February 21). In Wikipedia. https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology
Fernandez, B. (2019). Fake news in Asian politics. Global Risk Insights, 1.
Fernandez, M. (2021, February 6). Anti-vaccine activists emboldened in California. The New York Times. https://www.nytimes.com/2021/02/06/us/california-covid-vaccine.html. Last accessed 28 Feb 2021.
Food and Agricultural Organization of the United Nations. (2020, November 17). Communicating food safety in the era of COVID-19—Earning consumer trust. http://www.fao.org/asiapacific/news/detail-events/en/c/1330594/. Last accessed 28 February 2021.
Galeano, K., Galeano, R., & Agarwal, N. (2020). An evolving (dis)information environment—How an engaging audience can spread narratives and shape perception: A trident juncture 2018 case study. 253–265. https://doi.org/10.1007/978-3-030-42699-6_13
Galeano, K. K., Galeano, L. R., Mead, E., Spann, B., Kready, J., & Agarwal, N. (2019). The role of YouTube during the 2019 Canadian Federal election: A multi-method analysis of online discourse and information actors. Queen’s University.
Grant, M. (2004). Greek and Roman historians: Information and misinformation. Routledge.
Horton, R. (2020). Offline: Managing the COVID-19 vaccine infodemic. Lancet (london, England), 396(10261), 1474.
Jerit, J., & Zhao, Y. (2020). Political misinformation. Annual Review of Political Science, 23(1), 77–94.
Kearney, M., Chiang, S., & Massey, P. (2020). The Twitter origins and evolution of the COVID-19 “plandemic” conspiracy theory. Harvard Kennedy School, Misinformation Review (9). https://misinforeview.hks.harvard.edu/article/the-twitter-origins-and-evolution-of-the-covid-19-plandemic-conspiracy-theory/. Last accessed 28 Feb 2021.
Kermack, W. O., & McKendrick, A. G. (1927). A contribution to the mathematical theory of epidemics. Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character, 115(772), 700–721.
Lee, J. (2021, January 11). The utter familiarity of even the strangest vaccine conspiracy theories. The Atlantic. https://www.theatlantic.com/ideas/archive/2021/01/familiarity-strangest-vaccine-conspiracy-theories/617572/. Last accessed 28 Feb 2021.
Loomba, S., de Figueiredo, A., Piatek, S., de Graaf, K., & Larson, H. J. (2020). Measuring the impact of exposure to COVID-19 vaccine misinformation on vaccine intent in the UK and US. medRxiv.
Maleki, M., Mead, E., Arani, M. & Agarwal, N. (2021). Using an epidemiological model to study the spread of misinformation during the black lives matter movement. International Conference on Fake News, Social Media Manipulation and Misinformation (ICFNSMMM). Athens, Greece.
Marcoux, T., Mead, E., & Agarwal, N. (2020). The ebb and flow of the COVID-19 misinformation themes. Presented at the SBP-BRiMS.
Mason, L. E., Krutka, D., & Stoddard, J. (2018). Media literacy, democracy, and the challenge of fake news. Journal of Media Literacy Education, 10(2), 1–10.
McCourt School of Public Policy. (2020, September 24). Misinformation and the impact of social media in elections. Georgetown University. https://mccourt.georgetown.edu/news/misinformation-and-the-role-of-social-media-in-elections/. Last accessed 28 Feb 2021.
Mihailidis, P., & Viotty, S. (2017). Spreadable spectacle in digital culture: Civic expression, fake news, and the role of media literacies in “post-fact” society. American Behavioral Scientist, 61(4), 441–454.
Miller, J. (2019, June 10). Democracy and misinformation: The Cold War and today. The Newsmagazine of the American Historical Association: Perspectives on History. URL: https://www.historians.org/publications-and-directories/perspectives-on-history/summer-2019/democracy-and-misinformation. Last accessed 28 Feb 2021.
Myers, S. A., Sharma, A., Gupta, P., & Lin, J. (2014). Information network or social network? The structure of the Twitter follow graph. Proceedings of the 23rd International Conference on World Wide Web, 493–498. https://doi.org/10.1145/2567948.2576939
Obadimu, A., Khaund, T., Mead, E., Marcoux, T. & Agarwal, N. (2021). Developing a socio-computational approach to examine toxicity propagation and regulation in COVID-19 discourse on YouTube. Information Processing and Management Special issue on Dis/Misinformation Mining from Social Media, 58(5). https://doi.org/10.1016/j.ipm.2021.102660
Palatino, M. (2019, September 26). Combatting disinformation in Asia Pacific: Intended—And unintended—Consequences. IFEX. https://ifex.org/combatting-disinformation-in-asia-pacific-intended-and-unintended-consequences/. Last accessed 28 Feb 2021.
Pavlopoulos, J., Thain, N., Dixon, L. & Androutsopoulos, I. (2019). ConvAI at SemEval-2019Task 6: Offensive language identification and categorization with perspective and BERT. Proceedings of the 13th International Workshop on Semantic Evaluation, 571–576. https://doi.org/10.18653/v1/S19-2102
Reiss, D. (2021, January 20). COVID-19 vaccine misinformation and the anti-vaccine movement. Harvard Law, Petrie-Flom Center. https://blog.petrieflom.law.harvard.edu/2021/01/20/covid-19-vaccine-misinformation-anti-vaccine-movement/. Last accessed 28 Feb 2021.
Salzberg, S. (2020, February 3). How anti-vax activists use conspiracy theories to spread fear of vaccines. Forbes. https://www.forbes.com/sites/stevensalzberg/2020/02/03/how-the-anti-vaccine-cult-spreads-its-message/?sh=6318da5c2036. Last accessed 28 Feb 2021.
Sanovich, S. (2017). Computational propaganda in Russia: The origins of digital misinformation. S. Woolley & P. N. Howard (Eds.), Working Paper 2017.3. Project on Computational Propaganda. comprop.oii.ox.ac.uk. 32 pp.
Sanovich, S., Stukal, D., & Tucker, J. A. (2018). Turning the virtual tables: Government strategies for addressing online opposition with an application to Russia. Comparative Political Studies, 50(3), 435–482.
Segerberg, A., & Bennett, L. (2013). The logic of connective action: Digital media and the personalization of contentious politics. Cambridge University Press. https://doi.org/10.1017/CBO9781139198752
Seitz, A. (2021, February 24). Fake accounts gain traction as they praise China, mock US. AP NEWS. https://apnews.com/article/media-social-media-coronavirus-pandemic-covid-19-pandemic-china-7339598fed868fcfe109999bf071a77c. Last accessed 28 Feb 2021.
Shahi, K.G., Dirkson, A., & Majchrzak, T. A. (2020). An exploratory study of COVID-19 misinformation on Twitter. arXiv e-prints, arXiv-2005.
Shao, C., Ciampaglia, G. L., Flammini, A., & Menczer, F. (2016). Hoaxy: A platform for tracking online misinformation. 745–750. https://doi.org/10.1145/2872518.2890098
Shu, K., Sliva, A., Wang, S., Tand, J., & Liu, H. (2017). Fake news detection: Network data from social media used to predict fakes. ACM SIGKDD Explorations Newsletter, 19(1), 22–36.
Søe, S. O. (2018). Algorithmic detection of misinformation and disinformation: Gricean perspectives. Journal of Documentation, 74(2), 309–332.
Soll, J. (2016, December 18). The long and brutal history of fake news. Politico Magazine. https://www.politico.com/magazine/story/2016/12/fake-news-history-long-violent-214535/. Last accessed 28 Feb 2021.
Spann, B., Agarwal, N., Johnson, O., & Mead, E. (2020, October 18–21). Modeling protester orchestration through connective action: A COVID-19 lockdown protest case study. 2020 International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction, and Behavior Representation in Modeling and Simulation (SBP-BRiMS 2020). Washington, DC.
Starmans, B. (2018). 10 examples of fake news from history. The Social Historian. https://www.thesocialhistorian.com/fake-news/. Last accessed 28 Feb 2021.
Tsfati, Y., Boomgaarden, H. G., Strömbäck, J., Vliegenthart, R., Damstra, A., & Lindgren, E. (2020). Causes and consequences of mainstream media dissemination of fake news: Literature review and synthesis. Annals of the International Communication Association, 44(2), 157–173.
Valenzuela, S., Halpern, D., Katz, J. E., & Miranda, J. P. (2019). The paradox of participation versus misinformation: Social media, political engagement, and the spread of misinformation. Digital Journalism, 7(6), 802–823.
Watts, C. (2018). Messing with the enemy: Surviving in a social media world of hackers terrorists, Russians, and fake news. Harper Business.
Weitz, R. (2020). ICDS Diplomaatia magazine, November 13, 2020. Assessing the Russian Disinformation Campaign During COVID-19. https://icds.ee/en/assessing-the-russian-disinformation-campaign-during-covid-19/. Last accessed 22 Feb 2021.
Wendling, M. (2018). The (almost) complete history of ‘fake news’. BBC. https://www.bbc.com/news/blogs-trending-42724320. Last accessed 28 Feb 2021.
Wilson, T., & Starbird, K. (2020). Cross-platform disinformation campaigns: Lessons learned and next steps. Harvard Kennedy School: Misinformation Review, 1(1), 2020.
Zhang, X., & Ghorbani, A. A. (2019, August). An overview of online fake news: Characterization, detection, and discussion. Information Processing and Management (2018), 102025. https://doi.org/10.1016/j.ipm.2019.03.004
Acknowledgements
This research is funded in part by the U.S. National Science Foundation (OIA-1946391, OIA-1920920, IIS-1636933, ACI-1429160, and IIS-1110868), U.S. Office of Naval Research (N00014-10-1-0091, N00014-14-1-0489, N00014-15-P-1187, N00014-16-1-2016, N00014-16-1-2412, N00014-17-1-2675, N00014-17-1-2605, N68335-19-C-0359, N00014-19-1-2336, N68335-20-C-0540, N00014-21-1-2121), U.S. Air Force Research Lab, U.S. Army Research Office (W911NF-20-1-0262, W911NF-16-1-0189), U.S. Defense Advanced Research Projects Agency (W31P4Q-17-C-0059), Arkansas Research Alliance, the Jerry L. Maulden/Entergy Endowment at the University of Arkansas at Little Rock, and the Australian Department of Defense Strategic Policy Grants Program (SPGP) (award number: 2020-106-094). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding organizations. The researchers gratefully acknowledge the support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Agarwal, N., Mead, E., Spann, B., Donoven, K. (2022). Developing Approaches to Detect and Mitigate COVID-19 Misinfodemic in Social Networks for Proactive Policymaking. In: Gill, R., Goolsby, R. (eds) COVID-19 Disinformation: A Multi-National, Whole of Society Perspective. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-030-94825-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-94825-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-94824-5
Online ISBN: 978-3-030-94825-2
eBook Packages: Social SciencesSocial Sciences (R0)