1 Introduction

In an increasingly linked world where information moves swiftly and reaches a vast number of people, social media has developed dramatically in recent years. Currently, social media data are growing explosively, they are becoming more and more complicated and diversified, and this is due to the vast population of social media users, as well as the sheer volume of user-generated content (Gabr et al. 2021).

Social media has allowed any individual with access to the internet to publicly express their opinion, to be free to share and publish whatever they want (El Alaoui et al. 2019). This kind of liberty creates various data issues that can lead to a degraded quality of data, and it seems plausible that any poor quality of data could have a negative impact. Typically, it is widely known that data quality problems, such as incompleteness, redundancy, and inconsistency, make useless the overall process of using and processing this data. Therefore, getting a high level of data quality is deemed to be one of the most significant challenges, this level of quality is assessed using various dimensions and quality metrics. According to (Alizamini et al. 2010), data will be of high quality, if it is suitable for decision making, applications, and planning. Inspecting data to assess its present quality and the possible scope of any data quality issues is the goal of analyzing the quality of social media data (Reda and Zellou 2022; Woodall and Parlikad 2010). This paper’s goal is to outline how to measure and assess the quality of social media data using a set of quality metrics already defined during the construction of the SMDQM model presented in Reda and Zellou (2022). For this, a thorough review of the literature is conducted to collect existing quality models in order to identify the most commonly used dimensions needed to assess the quality of social media data in a reliable way. In this study, we introduce a novel quality model for the assessment of social media data quality, in which we select eight quality metrics which are credibility, timeliness, popularity, relevancy, accessibility, reliability, presentation and accuracy. We suggest modeling data quality metrics with fuzzy logic to address imprecision and subjectivity, allowing us to describe data quality requirements using a set of linguistic expressions on quality measurements. Then, we perform extensive experiments to evaluate the SMDQM model; to do that, we chose Twitter because it is one of the most widely used social media platforms for consumers to share their opinions, complaints, concerns and compliments about a product or service, as well as because of its attributes that are significant for data analysis.

This paper is structured according to the following: Section 2 defines the basic terms used in this work, and the state-of-the-art of data quality, quality dimensions and metrics, quality model and quality assessment. Section 3 presents the background of fuzzy logic. A detailed description of the implementation of the proposed model SMDQM is presented in Sect. 5. Section 6 presents experimental results. Finally, Sect. 7 concludes the paper by mentioning our future directions to approach them in the coming up works.

2 Preliminaries

2.1 Data

In the literature, various views exist regarding the definition of data, such as the raw materials needed for information, or as a set of facts. Elmasri and Navathe (2000) defined data as facts with implicit meaning that may be measured, recorded, and documented. According to Lee et al. (2006), data can be considered as a valuable asset. Typically, an asset can be either tangible, such as place or location, or intangible, such as knowledge and methodologies. Therefore, a data is an elementary description of a reality or an abstract notion, which has not yet been interpreted and put into context. In general, there are three types of data to consider: structured, semi-structured and unstructured data.

  • Structured data are based on a predefined schema and conform to particular specifications (Sint et al. 2009). It can be stored in relational database system and thus fit into a clearly defined data model or structured files (i.e., csv).

  • Semi-structured data are a kind of structured data but does not have the strict structure of the data model. They are frequently described as terms without schema or self-describing, without a distinct description of the data type or structure such as web page and XML file (Sint et al. 2009).

  • Unstructured data lack a standardized identifiable form. Photographs and graphic images, videos, streaming data, and files, for example, cannot be stored in rows and columns in a relational database (Sint et al. 2009).

2.2 Quality

The quality is defined as the set of characteristics, properties that make something correspond well or poorly to its nature [9]. The International Organization for Standardization (ISO) defines it as all of an entity’s attributes that confer the capacity to meet explicit and implicit standards [10].

2.3 Data quality

In data quality literature, there is no precise definition of the term data quality. However, a widely accepted way of defining data quality is as fitness for use (Tayi and Ballou 1998). it is considered as the degree to which the data meets the user’s needs or is suitable for a specific process and depends hence on the context of use and the requirements of the users (Berti-Equille 1999).

Data quality is defined by the International Organization for Standardization (ISO) as the degree to which a certain collection of intrinsic characteristics satisfies a specified requirement or expectation, often inferred or required (Hoyle 2006), while (Deming 1982) and Juran (2003) defined it as the ability to be used in the context of decision making, activities, and planning, and it is considered to be directly related to context.

Generally, the quality of data is often defined as the adequacy of a given data and its properties for a specific use case, depending on the data consumer who uses it (Nikiforova 2020). It is conceived as a multidimensional concept and, over time, researchers have proposed various definitions, such as “user satisfaction” (Wayne 1983) and as “conformity to requirements” (Crosby 1979). According to Olson (2003), the concept of data quality can refer to the adequacy of data to fulfill the intended goal. It depends on the requirements that the user expects to execute or the data value that the user expects to obtain. This concept involves several dimensions, as different aspects have to be taken into account. These aspects are modeled using data quality dimensions that are assessed by certain metrics (Ehrlinger and Woss 2018).

2.4 Quality dimensions and metrics

The quality dimensions are an important part for assessing data quality, as these data quality can be evaluated using several dimensions and metrics. A quality dimension refers to a property of data quality that represents an aspect of these data and can be used to guide the process of understanding quality (Laranjeiro et al. 2015). According to Wang and Strong (1996), data quality dimension is a set of data quality attributes that represent a unique aspect of data quality, i.e., each dimension concerns a specific aspect.

Managing data quality involves selecting a set of dimensions within a particular context, and then classifying them in order to establish a quality model with which we can evaluate the quality of our data and assess their relevance (Reda et al. 2020). Several dimensions of data quality are addressed in the literature, but the most commonly used are accuracy, timeliness, completeness, and consistency. Examples of quality dimensions are illustrated in Table 1.

Table 1 Examples of data quality dimensions
Full size table
Table 2 Examples of data quality metrics
Full size table

A data quality metric is a quantifiable resource that defines how a dimension is measured (Reda and Zellou 2023). Within each quality model, a set of specific quality dimensions, quality sub-dimensions, quality measures, and the relationships between these dimensions and measures are described. The quality metrics defined in a quality model provide information about the dimensions and sub-dimensions of quality, providing relevant details around quality measures, such as definitions, formulas or scales, and generally a classification of various quality metrics is included in the quality model (Radulovic et al. 2018). In other words, quality metrics may be viewed as formulas for quantifying the fulfillment of dimensions in numerical values. While a quality dimension is a wide idea in general, a related metric helps to establish a specific meaning for the dimension. As a result, many metrics may be associated with the same quality dimension, and their application will assess a variety of aspects of the dimension (Arolfo et al. 2020). Table 2 shows several examples of quality measures.

2.5 Quality model

The goal of a data quality model is to provide a definition of the various data quality characteristics within a hierarchy, in order to know the level of quality of the data included in different systems, as well as to help the user create his own quality profile. Hence, a quality model is used as a reference to the quality measures to be evaluated (Reda and Zellou 2022). By specifying detailed quality measures, such as formulas, definitions, or scales, quality models indicate which measures are useful for evaluation and how they should be measured (Hitzler et al. 2016). Various researchers described data quality models as invaluable resources for quality assessment and, accordingly, they serve as a reference for the quality measures to be evaluated (Even and Shankaranarayanan 2009).

2.6 Data quality assessment

According to ISO definition, quality assessment describes the overall evaluation of data quality using established quality criteria. It might be the outcome of a quality scoring or grading procedure, either quantitative or qualitative scoring may be used [31]. Data quality assessment refers to the quality evaluation of meaningful data, where the context and intended use of the data are taken into account (Caballero et al. 2009). Typically, it entails the development of metrics to test and identify the present level of data quality; this evaluation is regarded as a component of data quality management (Reda and Zellou 2022). The first step toward any quality assessment is to define the quality requirements, which are based on data users and their needs. It then consists of measuring the relevant dimensions or quality criteria and comparing the results of this evaluation with the user’s quality requirements (Radulovic et al. 2018).

Fig. 1
figure 1

Overview of the operation of fuzzy system

Full size image
Fig. 2
figure 2

SMDQM model process

Full size image

3 Fuzzy logic background

It is frequently challenging to give precise definitions or descriptions of certain concepts and the relationships between them, especially in the context of data quality. Fuzzy sets are used in our study to reflect the quality of the data in order to tackle this issue.

The concept of fuzzy logic introduced by ZADEH (1965) has been implemented in various scientific and technological sectors. Fuzzy sets have been developed to represent and operate imperfect data while tolerating different kinds of uncertainty. Typically, there are three major processes involved: Fuzzification, fuzzy rules, and defuzzification are the first three steps. Figure 1 visualizes the overview of operation of fuzzy systems.

A fuzzy set is characterized by a membership function defined on the universe of discourse. This function specifies the degree to which a given element belongs to the fuzzy set by mapping it to a range spanning the interval [0,1]. When the membership function has a value of 0, it signifies that the associated element is not a part of the fuzzy set. When it has a value of 1, it fully belongs to the set. Values between 0 and 1 represent fuzzy members, which only partially belong to the fuzzy set (ROSS 2012). The formal definition of a fuzzy set A is:

$$\begin{aligned} A = \{ (x, \mu _A(x)) | x \in X \} \end{aligned}$$
(1)

where X is the discourse universe, \(\mu _A\) is the membership function representing a fuzzy set A, and \(\mu _A(x)\) is the membership degree of the element x in the fuzzy set A (\(\mu _A(x)\) specifies how much x is compatible with the set A).

The fuzzification process is constructed using a variety of membership functions, both linear and nonlinear. This step quantifies the degree to which each input belongs to different categories (ROSS 2012).

The rule base, often referred to as fuzzy if-then rules, is included into language terms that are created by experts or that are based on data set extractions. Every rule is made up of a mathematical procedure that converts professional knowledge into murky if-then rules. The antecedent (the if section) and the consequent (the then part) are the two components of fuzzy rules (WANG et al. 2009).

In defuzzification step, the fuzzy output sets are converted back into crisp values or decisions. Various defuzzification methods can be used, such as centroid, height, or bisector, depending on the specific application.

4 SMDQM

In our previous study, as described in Reda and Zellou (2022), our objective was to develop a social media data quality model aimed at assessing the quality of data collected from various social media platforms. To achieve this, we proposed a comprehensive five-step methodology, as depicted in Fig. 2. The proposed model encompasses eight key quality dimensions, which were derived from a compilation of 60 dimensions collected from 13 different research sources. These eight dimensions were identified as the most prominent and significant factors based on their prevalence and relevance in the existing literature.

Fig. 3
figure 3

Presentation of timeliness dimension

Full size image
Fig. 4
figure 4

Presentation of popularity dimension

Full size image

By establishing this social media data quality model, we sought to offer researchers and practitioners a robust framework to evaluate and ensure the reliability, accuracy, and usability of social media data. The model’s effectiveness was validated through a rigorous evaluation process, and it holds great potential in advancing research in various domains reliant on social media data analysis.

5 FULMQA: fuzzy Logic-based model for quality assessment

In the following sections, we describe in greater depth and provide quality metrics for each dimension of the novel data quality model, which includes credibility, timeliness, popularity, relevancy, accessibility, reliability, presentability, and accuracy.

5.1 Timeliness

Over time, correct data may become incorrect due to changing circumstances. Timeliness dimensions is defined in Cai and Zhu (2015) as the difference in time between the generation and acquisition of data and its use. It refers to the freshness of the data, and represents the measurement of whether the data is up-to-date enough for the given task (Chai et al. 2009; Immonen et al. 2015). A post closer to the time of the search, the more certain we are that the data are relevant (Reuter et al. 2015). The timeliness can be assessed by calculating the age metric which measures the age of a post; it is calculated by the difference between the current timestamp \(T_{now}\) and the time when the post was created \(T_{post}\), with the following equation:

$$\begin{aligned} Age_{post} = T_{now} - T_{post} \end{aligned}$$
(2)

Given \(T_0\) which represents the age of the most recent post, and \(T_t\) the age tolerable, the timeliness of post p is calculated by:

$$\begin{aligned} T(p) = {\left\{ \begin{array}{ll} 1, &{} \text {if } Age_{post} \le T_0 \\ \frac{T_t - Age_{post}}{T_t - T_0}, &{} \text {if } T_0< Age_{post} < T_t \\ 0, &{} \text {if } Age_{post} \ge T_t \end{array}\right. } \end{aligned}$$
(3)

When we have a recent post, we are more assured that the timeliness is high, i.e., the more recent the post, the more timeliness is high. Figure 3 shows the presentation of timeliness dimension.

Fig. 5
figure 5

Presentation of credibility dimension

Full size image

5.2 Popularity

Popularity dimension can be defined by the use and reuse of a resource; this extensive use tends to lead to greater trust. When an author posts accurate data, that author has a particular number of followers, or the data are liked and then shared or circulated by other people (Immonen et al. 2015). According to the social context, the popularity can be assessed using a lot of criteria such as the number of followers, number of friends, number of comments, number of likes, number of retweets, and number of share. Based on several studies (Alrubaian et al. 2017), the criteria number of shares/retweets is considered to be one of the best indicators used to measure the popularity, because it can be very attractive for the user to use a post which has been shared or retweeted many times. To measure popularity, we use \(S_p\) to denote the number of shares or retweets of a post P, \(S_0\) is the number of shares/retweets of the post that has the highest number of shares/retweets, and \(S_t\) the tolerable number of shares/retweets. Popularity can be calculated as follows:

$$\begin{aligned} P(p) = {\left\{ \begin{array}{ll} 1, &{} \text {if } S_p \ge S_0 \\ 1- \frac{S_0 - S_p}{S_0 - S_t}, &{} \text {if } S_t< S_p < S_0 \\ 0, &{} \text {if } S_p \le S_t \end{array}\right. } \end{aligned}$$
(4)

The degree of popularity increases with the increasing number of the criteria given to a particular post or author, i.e., with the increasing number of shares/retweets of the post. Figure 4 shows the presentation of popularity dimension.

5.3 Credibility

Evaluating the credibility of social media data is a topic of broad and current interest. Whenever we refer to the credibility of data, we are directly asking whether this data is supposed to be believed or not, which means, if the data are credible, so we can believe it, whether it is real or not. According to Salvatore et al. (2020), it is about the different objective and subjective parts of the believability of a source or a message. Several research propose the assessment of credibility at user, and content levels (Gupta et al. 2019; Yang et al. 2019; Verma et al. 2019). However, in our study, we suppose that a credible users provide credible contents, so users with high-credibility are more likely to share believable contents.

Credibility of users can be measured based on the number of followers/friends of a user. For instance, a lot of users are likely to believe a Twitter user with many followers. We use \(F_p\) to denote the number of followers/friends of the user created the post P, and \(F_0\) the number of followers/friends given to the user with the highest number of followers/friends and \(F_t\) the number of followers/friends tolerable. Credibility can be measured as follows:

$$\begin{aligned} C(p) = {\left\{ \begin{array}{ll} 1, &{} \text {if } F_p \ge F_0 \\ 1- \frac{F_0 - F_p}{F_0 - F_t}, &{} \text {if } F_t< F_p < F_0 \\ 0, &{} \text {if }F_p \le F_t \end{array}\right. } \end{aligned}$$
(5)

The degree of credibility increases with the increasing number of followers/friends given to a particular author. Figure 5 shows the presentation of credibility dimension.

Fig. 6
figure 6

Presentation of reliability dimension

Full size image

5.4 Reliability

In a social media context, all people can publish whatever they like and any kind of information, whether truthful or not. When we talk about data reliability, we logically imply whether the data are trustworthy or not; this means, if the data is reliable, we can trust it. For example, people can rely on the opinions of others through Twitter posts to know the quality of a movie; similarly, if a YouTube video has received a lot of views, many people may trust it. According to Chai et al. (2009), the reliability describes the degree to which the data is trustworthy and correct. There are some authors who frequently link reliability with four other dimensions like verifiability, credibility, popularity and reputation (Firmani et al. 2015; Arolfo et al. 2020). However, no social media platform, including Twitter, has the ability and want to verify all their users, so it is expected that the majority of users in social media are unverified. Based on this, we consider the following dimensions to define the reliability:

$$\begin{aligned} Reliability = Credibility \vee Popularity \vee Reputation\nonumber \\ \end{aligned}$$
(6)

Since we already evaluated the popularity and credibility in our study, we will focus on the reputation which is defined as a judgment done by a user to decide the Trustworthiness of a source (Arolfo et al. 2020). The reputation can be evaluated for example based on the number of followers/friends of an account, number of likes, registration date, or number of posts from the same author. We use \(L_p\) to indicate the number of likes given to the post P, and \(L_0\) the number of likes of the post that has the highest number of likes, and \(L_t\) is the number of likes tolerable. The reputation can be calculated as follows:

$$\begin{aligned} Rep(p) = {\left\{ \begin{array}{ll} 1, &{} \text {if } L_p \ge L_0 \\ 1- \frac{L_0 - L_p}{L_0 - L_t}, &{} \text {if } L_t< L_p < L_0 \\ 0, &{} \text {if }L_p \le L_t \end{array}\right. } \end{aligned}$$
(7)

Using the method proposed by (Larsen 1980), the final output of the reliability Rel(p) is the weighted average of all outputs of the 3 metrics (aggregation) as follows:

$$\begin{aligned} Rel(p) = \frac{m_1 C(p) + m_2 P(p) + m_3 Rep(p)}{3} \end{aligned}$$
(8)

where \(m_1\), \(m_2\), \(m_3\) are the coefficients. The degree of reliability increases with the increasing number of followers/friends, likes, and shares/retweets given to a particular post or author. Figure 6 shows the presentation of reliability dimension.

5.5 Relevancy

Generally, the relevancy dimension is related to the gains that a user can achieve by using the data; it is used to describe the extent to which these data are useful and applicable for the purpose (Chai et al. 2009; Immonen et al. 2015). In other words, it is the degree to which the data produced match the needs of the users (Salvatore et al. 2020). The relevancy could be measured by detecting a sentiment from the post, whether positive, negative or neutral. Typically, there are different methods for calculating the sentiment of a text. A sentiment analysis, for example, is used to derive meaning from text. It is one of the most widely used text classification techniques for analyzing and categorizing the sentiment underlying a communication, whether positive, negative, or neutral (Salvatore et al. 2020). In this study, we will calculate sentiment score by categorizing and counting the amount of negative and positive words from the supplied post, and then taking the ratio of the difference between positive and negative word counts and total word count. The following equation is used to determine sentiment score:

$$\begin{aligned} Sentiment = \frac{Positive_{words} - Negative_{words}}{Total_{words}} \end{aligned}$$
(9)

If Sentiment score > 0, it is a positive post, if Sentiment score < 0, it is a negative post, and if Sentiment score \(=\) 0, it is a neutral post. Therefore, if a post expresses a positive or negative feeling, it will be considered as a relevant post, and if the sentiment is neutral, i.e., no sentiment could be computed by a Natural Language Processing (NLP) tool, it will be considered not relevant. Let \(S_p\) be the score of post P, and \(S_0\) be the score associated with the post that has the highest score positive/negative, \(S_t\) is the tolerated score for the most relevant post positive/negative, and the relevancy can be calculated as follows:

$$\begin{aligned} R(p){=}{\left\{ \begin{array}{ll} 1, &{} \text {if } (S_p> 0 \text { and } S_p \ge S_0) \text { OR } (S_p< 0 \text { and } S_p {\le } S_0) \\ 1 {-} \frac{S_0 - S_p}{S_0 - S_t}, &{} \text {if } S_p> 0 \text { and } S_t \le S_p {\le } S_0 \\ \frac{S_t + S_p}{S_t + S_0}, &{} \text {if } S_p< 0 \text { and } S_t \ge S_p {\ge } S_0 \\ 0, &{} \text {if } (S_p > 0 \text { and } S_t \ge S_p) \text { OR } (S_p < 0 \text { and } S_t {\le } S_p) \end{array}\right. }\nonumber \\ \end{aligned}$$
(10)

The degree of relevancy increases with the increasing number of positive and negative words. Figure 7 shows the presentation of relevancy for negative and positive sentiment.

Fig. 7
figure 7

Presentation of relevancy for negative a and positive b sentiment

Full size image
Fig. 8
figure 8

Presentation of accessibility dimension

Full size image

5.6 Accessibility

The accessibility dimension is about the ability to retrieve the data, i.e., the ease of access. It could be closely correlated with how challenging users find it to access the data. In other words, it indicates how quickly and simply the data may be retrieved (Salvatore et al. 2020), and it also deals with the additional requirements to access the data, i.e., registrations that are required before accessing any social media platform. To determine the accessibility of a post, we must first determine whether or not users can find it, which can be accomplished by examining for instance the number of shares/retweets, likes, or comments. Specifically, if the post has been shared or commented on by several users, the post is now accessible to users. In our study, we use the numbers of comments to measure accessibility, we use \(C_p\) to indicate the number of comments given to the post p, and \(C_0\) the number of comments of the post that has the highest number of comments, and \(C_t\) is the number of comments tolerable. The accessibility of p can be calculated as follows:

$$\begin{aligned} Acc(p) = {\left\{ \begin{array}{ll} 1, &{} \text {if } C_p \ge C_0 \\ 1- \frac{C_0 - C_p}{C_0 - C_t}, &{} \text {if } C_t< C_p < C_0 \\ 0, &{} \text {if }C_p \le C_t \end{array}\right. } \end{aligned}$$
(11)

The degree of accessibility increases with the increasing number of comments given to a particular post. Figure 8 shows the presentation of accessibility dimension.

Fig. 9
figure 9

Presentation of presentability dimension

Full size image
Fig. 10
figure 10

Presentation of accuracy dimension

Full size image

5.7 Presentability

The presentability of data specifies how correct and error-free data are, and it is strictly linked to the concept of ‘errors’ which can affect the results of the analysis. According to Salvatore et al. (2020), presentability refers to the right interpretation of data that enables users to completely comprehend the information; it describes the level of correctness that assures the data is error-free. For instance, suppose when a manually written text is processed, and some of the characters are not clearly defined. In this regard, it is suggested to use the percentage of misspelled words as a gauge of the presentability of data. Given a post p, with W a collection of correct and non-correct words in the post p, we use Earley parser algorithm (Earley 1970) which receives as input two parameters: a grammar noted G and a list of words representing the post to check correct words.

$$\begin{aligned} Score = \frac{ \text {Number of misspelled words in P} }{\text {Total words in P} } \end{aligned}$$
(12)

Let \(S_p\) be the score of the post p, \(S_0\) is the score of the post that has the highest score, and \(S_t\) is the score tolerable. The presentability of p, denoted Pre(p), can be measured as follows:

$$\begin{aligned} Pre(p) = {\left\{ \begin{array}{ll} 1, &{} \text {if } S_p \ge S_0 \\ 1- \frac{S_0 - S_p}{S_0 - S_t}, &{} \text {if } S_t< S_p < S_0 \\ 0, &{} \text {if } S_p \le S_t \end{array}\right. } \end{aligned}$$
(13)

The degree of presentability increases with the increasing score given to each post. Figure 9 shows the presentation of presentability dimension.

5.8 Accuracy

Accuracy dimension concerns the degree of validity of data; it refers to the facility and the process of extracting insight from the data. In other words, it accuracy describes the closeness between two values, the correct representation of the real fact that the other aims to represent. According to Wang and Strong (1996), accurate data must be correct, relevant, and come from credible and reputable sources. In our study, we primarily focus on these three metrics to measure accuracy, we have previously evaluated the presentability to ensure that the data correct, relevancy and reliability are also evaluated for credibility and reputation. Based on this, we consider the following dimensions to define the accuracy:

$$\begin{aligned} Accuracy = Presentability \vee Relevancy \vee Reliability\nonumber \\ \end{aligned}$$
(14)

Using the method proposed by Larsen (1980), the final output of the accuracy A(p) is the weighted average of all outputs of the 3 metrics (aggregation) as follows:

$$\begin{aligned} A(p) = \frac{ c_1 Pre(p) + c_2 R(p) + c_3 Rel(p)}{3} \end{aligned}$$
(15)

where \(c_1\), \(c_2\), \(c_3\) are the coefficients. The degree of accuracy increases with the increasing number of presentability, relevancy, and reliability given to a particular post. Figure 10 shows the presentation of accuracy dimension.

Fig. 11
figure 11

A comprehensive design of the experiments

Full size image

6 Experiments and results

This section serves as a comprehensive exploration of the implementation of our new model, FULMQA, along with the details of the experiments conducted to assess its performance utilizing the metrics previously introduced. Additionally, it encompasses the reporting of the experiment outcomes and engages in an in-depth discussion of the findings. Twitter was selected as the social media platform for this evaluation, representing a pertinent and dynamic environment for evaluating tweet quality. The overarching objective of these experiments is to not only assess tweet quality, but also to provide a demonstrative and illustrative framework for evaluating the quality of tweets on social media platforms. Through these experiments, we aim to shed light on the methodologies and metrics employed to gauge the quality of tweets in a transparent and informative manner.

Table 3 Features for metrics computation
Full size table

6.1 Datasets

We utilized the Python Twitter API,Footnote 1 known as Tweepy, to access publicly available tweets and extract user-friendly content and user information. Our data collection process involved creating two distinct datasets, each centered around a different hashtag: #coronavirus and #cancer. One dataset comprised approximately 30 tweets. Regarding hashtag selection, the choice of hashtags was indeed a crucial component of our experimental design. These hashtags were selected due to their relevance and significance in contemporary social media discussions. We aimed to evaluate the impact of different hashtags on tweet quality, and this selection was a key variable in our methodology.

It is important to note that Twitter’s API imposes certain limitations on our queries. We are confined to a query length of 1000 characters, and we can retrieve historical data spanning roughly 6 to 10 days. Our experiment encompassed all eight metrics. To gain a clearer picture, Table 3 provides a breakdown of the specific features we utilized in the computation of these metrics.

Now, let us delve into the specifics of our experimental approach. Figure 11 visually outlines the step-by-step implementation of our evaluation model.

6.2 Implementation

The step-wise procedure of each metric is presented in the following algorithms. The algorithms 1, 2, 3, 4, 5, 6, 7, and 8 describe our assessment procedure for respectively timeliness, presentability, credibility, popularity, relevancy, reliability, accessibility, and accuracy. These algorithms serve as comprehensive guides to the systematic evaluation of each metric, ensuring transparency and reproducibility in our approach to assessing tweet quality.

Algorithm 1
figure a

Timeliness assessment algorithm

Full size image
Algorithm 2
figure b

Presentability assessment algorithm

Full size image
Algorithm 3
figure c

Credibility assessment algorithm

Full size image
Algorithm 4
figure d

Popularity assessment algorithm

Full size image
Algorithm 5
figure e

Relevancy assessment algorithm

Full size image
Algorithm 6
figure f

Reliability assessment algorithm

Full size image
Algorithm 7
figure g

Accessibility assessment algorithm

Full size image
Algorithm 8
figure h

Accuracy assessment algorithm

Full size image

6.3 Results and discussions

Once the experiment has been executed, the overall results are depicted in Fig. 12. This presentation encapsulates the overarching results, offering a concise and accessible summary of our research outcomes.

To assess the effectiveness of our metrics, we intentionally included tweets from users with varying follower counts, as well as tweets with diverse levels of likes, retweets, and comments across both datasets, ensuring that our evaluation model was subjected to a spectrum of social influence scenarios and encompass the full spectrum of user interactions within the social media ecosystem.

Additionally, it is worth noting that the two datasets were extracted at different points in time, as depicted in Fig. 12h. The timestamps reveal that the data for the cancer dataset was collected from “2022-11-10 17:09” to “2022-11-10 15:23,” whereas the earliest tweet in the coronavirus ataset was posted at “2022-11-10 16:49,” and the latest one at “2022-11-09 17:11.” This temporal contrast highlights an interesting observation: the timeliness of the cancer dataset is marginally superior to that of the coronavirus dataset. It implies that tweets within the cancer dataset are more contemporaneous and aligned with the timeline of our data collection efforts. This finding underscores the robustness of our model in effectively evaluating tweets across various temporal contexts. It is a testament to the model’s high performance and the quality of results it can produce, reaffirming its efficacy in assessing tweet quality across a spectrum of characteristics.

Fig. 12
figure 12

Representation of a reliability b accuracy c accessibility d credibility e popularity f presentability g relevancy and h timeliness metrics

Full size image

7 Conclusion and future work

In this paper, we presented the implementation of our new model FULMQA; FUzzy Logic-based Model for Quality Assessment; a quality model that is primarily designed to assess the quality of data derived from social media platforms. Our model includes eight data quality metrics namely: timeliness, credibility, reliability, presentation, popularity, relevancy, accessibility, and accuracy. To obtain a better understanding of how we assess the quality of data from social media, we run an extensive experiments to evaluate the performance of the model using a real implementation on two different datasets from 30 tweets. In the future work, we will propose a holistic model that includes other metrics.