Abstract
Trust in the news media received wide scholarly attention for almost a century, which was further boosted as a result of recent developments in the media landscape and changes in how news is made and consumed. Despite that, the conceptualization of trust in the news media is still debated, and its measurement comparability has not yet been established. In this paper, I build up on earlier conceptualizations of trust in the news media, and test three theoretically derived measurement models to determine their cross-cultural equivalence in 28 EU countries. Using Eurobarometer data, I test the validity and comparability of these measurements employing multi-group confirmatory factor analysis. The findings indicate that trust is a unidimensional latent construct, equally interpreted across contexts. People's level of trust in the news media reflects their general attitude to the news stories and reporters in all sources of media they are exposed to. While bifactorial measurements of news media trust, differentiating between legacy and online sources, have some merit in single case-studies, they are non-invariant and therefore non comparable. This means that any cross-population differences found employing them are likely a function of measurement idiosyncrasies or other unknown factors.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
It has long been recognized that the extent to which citizens trust political and social institutions is of vital importance to the stability and performance of democratic political systems and modern societies (Almond and Verba 1963; Hooghe 2011). Some scholars see trust in institutions as a prerequisite for the very survival of democratic regimes (Norris 1999), while others perceive it as a means for preventing seismic changes in the workings of representative democracies (Klingemann and Fuchs 1995). There is an agreement in the literature that trust in political institutions indicates that members of the political community ‘feel that their own interests would be attended to, even if the authorities were exposed to little supervision or scrutiny’ (Easton 1975, p. 447).
Trust in social institutions has been similarly acknowledged as essential for the functioning of a diverse, modern society, as a means for promoting social cohesion, integration and stability (Kohring and Matthes 2007). The importance of one institution, namely the news media, was recognized early on, as central to the functioning of democracy. It provides the means by which citizens become informed and learn about their community, its problems and the various, often competing ideas for addressing these problems. Having this information at their disposal, citizens can lead their lives in a free and self-governing fashion (Strömbäck et al. 2020). Furthermore, they need to have confidence that other members of their community are similarly informed. Unless they can trust the news media to deliver the required common knowledge, they are less likely to trust reasoned political decision-making, or to accept political decisions (Coleman 2012; Tsfati and Cohen 2005).
With an emerging sense of a crisis in public trust in the news media and other social institutions (Bogaerts and Carpentier 2013; Coleman 2012) and reports on their historically low levels (Fisher 2018), researchers must have a valid way of gauging people’s trust in this specific social institution, to allow their cross-time and cross-case comparisons to be meaningful. Short of such a valid measure, trust in the news media may be a construct conditioned by time/case context and insufficiently independent from other constructs. This issue is particularly relevant in today’s news media landscape, with the considerable shifts in how news is made and consumed through a myriad of legacy and alternative sources (Kiousis 2001). Establishing the cross-context equivalence of such a measure is a prerequisite for any comparative analysis employing it. Surprisingly, and despite nearly a century-long interest in trust in news media, there is no common definition of what precisely this concept entails (Fisher 2018), nor is there an agreement on its operationalization, the specific media sources it should pertain to, or their relative weight and overall metric dimensionality.
This means that determining the meaning of observed cross-case differences is still impossible, as comparing the levels of news media trust across individual cases can only be possible if they are measured validly and invariantly (Medina et al. 2009). If such equivalence is not established, any cross-context differences may reflect measurement idiosyncrasies or other unknown factors rather than actual variances, and would therefore be biased. The purpose of this paper is to develop a valid way to measure the level of trust that citizens of 28 EU member states have in the news media, by establishing its cross-context invariance, which would provide the required evidence to indicate that it captures identical aspects of the latent trust construct. Since previous, comparative research relies primarily on survey measures of trust in the news media, the specific motivation in this paper is to understand to what extent can standard ‘trust in media’ survey indicators represent a single construct of trust in the news media, that will be transferable across diverse societies.
The paper proceeds as follows. First, a review of the literature on trust in the news media will be presented, to identify various conceptualizations, operationalizations and analytical perspectives employed to measure this construct. Measurement invariance will be introduced next, as a method that enables determining the validity of different measures of the latent construct. The data and case selection employed in the analyses will be discussed next, followed by specifying three hypothesized measurement models. The analysis will be completed by testing the invariance of these competing measurements of trust in news media and will be followed by a discussion of the findings with concluding remarks and recommendations for future research.
Trust in news media
The concept of trust has received wide research attention across various disciplines, including political science, sociology, anthropology, psychology, philosophy, economics, marketing, management studies and more (Bauer 2021). Trust is one of the enabling forces identified as necessary for all social activity and interactions (Delhey and Newton 2003). At its core, trust denotes a relationship between a trustor, who is willing to engage in action based on positive expectations about the trustee, the recipient of trust (Barber 1983). Trust situations are inherently asymmetrical, as the trustee holds the resources or competences, and the trustor cannot acquire them without cost. This asymmetry leads to uncertainty and risk to the trustor, who must voluntarily relinquish control, sometimes beyond the level warranted by circumstances (Yamagishi and Yamagishi 1994). In sum, trust is the willingness of the trustor ‘to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party’ (Mayer et al. 1995, p. 712).
An additional layer of complexity is introduced when considering the news media as the object of people’s trust. This is because individuals may have varying levels of trust in news messages, sources and journalists, and news organizations or groups thereof, as well as a general perspective on the media as a whole. The asymmetry and risk are clear. News media users do not possess the competences to meticulously verify the credibility of news content and often try to find some clues to legitimate their trust and to mitigate the inevitable risk (Strömbäck et al. 2020; Tsfati and Cohen 2005). They do so, based on believing that the media will perform in line with their expectations (Hanitzsch et al. 2018).
People’s trust in the media received scholarly attention for the better part of the twentieth century (Kiousis 2001). The interest only increased at the turn of the century, with global technological, economic and social changes, and even further with the move to Web 2.0, which introduced user-generated information and an unprecedented competition for people's attention from a myriad of alternative news sources (Newman et al. 2018; Strömbäck et al. 2020). Research on media trust regularly uses the terms trust and credibility interchangeably (Kiousis 2001; Kohring and Matthes 2007). This research can be grouped into three groups, based on the leading perspectives on the media’s performance at the micro, mezzo and macro levels.
Earlier research took a micro-level perspective and focused on people’s trust in the source of information. Pioneering research into source credibility conducted at Yale was based on earlier research on prestige (Arnett et al. 1931). It tried to establish that on top of the content of any message, the source that delivers it determines how it will be received by the public. The main finding was that perceptions of expertise and trustworthiness of the source, whether a person or an organization, influenced people's willingness to trust the message and change their minds on various issues (Hovland et al. 1953). Focusing on the source was criticized for lacking a robust theoretical base, as the Yale team failed to specify the core dimensions that cause people’s trust (Kiousis 2001). It was further unclear whether attributes identified in their research such as ‘expertness’ were trust dimensions or merely its correlates (Kohring and Matthes 2007).
Another micro-level line in research has explored the credibility of messages regardless of their source. This focus is preferred by many communication-studies scholars who view people's trust in the news media as their affirmation that news media reporting is done professionally, fully, accurately and fairly (Strömbäck et al. 2020; Tsfati and Cappella 2003). Various attributes have been identified as the basis for message trust, including safety, qualification and objectivity (Kiousis 2001), completeness, conciseness, consistency, representativeness, accuracy, authenticity and believability (Appelman and Sundar 2016). Other analyses have found that credibility of messages is associated with their structure and content (McCroskey and Mehrley 1969; Metzger et al. 2003).
The problem with this perspective is that it is similarly unclear whether these attributes are independent and comparable across different contexts. It is also unclear to what extent can the message be separated from the source that delivers it and the medium it is delivered through (Metzger et al. 2003). In addition, is it reasonable to expect the public to be able to determine the quality of news media reporting and to decide whether or not it is trustworthy (Strömbäck et al. 2020)? If people do possess these abilities, then why is the media needed at all? And what precisely is the risk these informed citizens take in trusting the news media? Lastly, if people constantly engage in a calculus to assess the quality of every piece of information reported by the media, we should expect a high volatility in any measure of media trust, especially considering the rich modern media landscape. Yet media trust is stable across time (Ladd and Podkul 2019), which indicates that it is not limited to specific sources or messages.
At the mezzo-level, researchers have tried to understand media credibility by understanding public trust in a particular news medium, such as newspapers, radio or television. Already in the 1930s, when the print industry became concerned with the effects of the popularization of news radio on readership and advertising revenues, it started studying and comparing different media credibility (Self and Roberts 2019). In the 1950s, it was the radio industry that became wary of the spread of television and conducted similar studies to assess and compare the credibility of different media (Newhagen and Nass 1989). In the 1970s, the US Television Information Office commissioned surveys about the credibility of different media, assuming their messages were inconsistent (Roper 1969). In recent years, many surveys collected data on attitudes toward online news, social media and video hosting platforms to complement the battery of media items and understand people’s attitudes toward them (Newman et al. 2018).
While this information is useful, it has three main shortcomings. First, it is unclear whether citizens’ trust in each medium is independent of the trust they have toward the others. In a modern, complex media landscape, it is hard to determine how the attitudes toward a particular medium can be disentangled from attitudes toward others. Second, there is no evidence that people actually possess specific opinions about each medium, and that they are equally thought through. It would be more reasonable to expect that their trust levels would be generally directed at the entire media they use as their source of information. Lastly, it is also unclear how comparable these measures are. For instance, will a 10-point drop in trust in one medium be equivalent to a 10-point drop in another? Will this similarity hold across context and time? And more fundamentally, is there a baseline level of trust for each of these media?
This division helps in more than merely identifying the unit of analysis of the trust object. It also reflects the envisioned societal role of the news media. If it is understood as being limited to delivering professional and accurate information to large audiences, then a micro- or mezzo-level perspective is in order. In contrast, if it is seen as a social institution, a wider perspective is required. The news media is the fourth estate, part of the democratic infrastructure, which includes institutions and processes that enable citizens to effectively participate in democracy (Coleman 2012; Exoo 2009). More evidence in support of this view is in the strong and persistent correlation between trust in the news media and trust in political institutions (Carr et al. 2014). Yet if trust in the news media only expresses approval of the integrity of sources, stories or the media that broadcasts them, it should be independent of approval of political institutions’ performance.
The solution proposed in this paper is to shift to the macro-level perspective and understand the news media as a social institution. Similar to other institutions, citizens use different aspects of the news media, sometimes many of them combined, and their attitudes toward the news media is based on how much trust they have in any combination of stories, sources and media they are exposed to, whether legacy or new. Measurement invariance of three hypothesized models will be tested. The first will group all media sources together and test whether they all load consistently into a single factor. The remaining two will test a hypothesized bifactorial structure, where media sources are partitioned into legacy versus online sources and news-producing versus web 2.0 apps used to disseminate news produced by other sources. Measurement invariance of any of these hypothesized models will be evidence for its cross-context comparability, and provide a valid proxy for the general public trust in news media.
Measurement invariance (MI)
The ability to compare across context is a bedrock principle in cross-cultural research, which typically engages with comparing the attitudes, opinions and behaviors of heterogenous groups of individuals, grouped along spatial, time, class or cohort lines. Establishing comparability is necessary to avoid systematic bias due to group membership (Bollen 1989). The central aim in testing for MI is to determine whether a measurement model possesses the same psychometric properties across heterogeneous contexts (Vandenberg and Lance 2000). In other words, whether the same factorial structure underlies a set of manifest variables and latent construct(s), and whether equivalent associations between different latent constructs and observed variables are equivalent across different times and cases (Davidov et al. 2014; Wu and Estabrook 2016).
To test the measurement equivalence of trust in the news media, I employed multiple-group confirmatory factor analysis (Bollen 1989; Jöreskog 1971). MGCFA is a useful tool in the family of structural equation modelling, employed to assess the validity of measurements of latent constructs across contexts (Bollen 1989; Vandenberg and Lance 2000). Construct validity will be established by exploring the variation and co-variation of trust in different media, and identifying the number and nature of latent variable(s) that account for this variation. Unlike other methods (e.g., exploratory factor analysis) which are explorative in nature and aim at identifying inter-correlation between indicators as well as the minimal number of unique factors to explain these correlations, MGCFA is a theory-guided process, designed to allow testing hypotheses and meaningfully compare measures. Using MGCFA will allow assessing the equivalence of any measure of trust in the news media and subsequently determine its cross-case comparability.
Measurement invariance is established in three steps. First, the configural invariance of the model must be validated. This type of equivalence is sometimes referred to as ‘pattern’ or ‘baseline’ invariance (Steinmetz et al. 2009). Configural invariance means that the overall factor structure stipulated by the measure, fits well for all sample groups. In simple terms, a measurement would be configurally invariant if the same survey items would be useful in measuring the latent construct and load on the same factors across all administrations (Vandenberg and Lance 2000). A configurally invariant measure of trust in the news media would mean that the basic meaning and structure of trust in different news media—legacy as well as online—exists in all sample countries and loaded into an identical factorial structure. If measures are found non-invariant configurally, it may reflect cross-context differences in the numbers of factors item loading scores across cases, which would indicate that it cannot be used comparatively. In contrast, under configural invariance, we should expect that citizens’ judgements of the various stories, sources and media to be strongly related to each other, and that together they would adequately capture citizens’ latent trust in the news media.
While factors may be structured similarly, the magnitude of their loadings, i.e., their individual contribution to the latent construct, may vary with context, which would make it impossible to compare the results across cases. Therefore, the next step will be to test the model’s metric invariance (or ‘weak invariance’), which builds upon configural invariance by requiring, in addition to the construct being measured by the same items, that the factor loadings of those items will be identical across cases (Brown 2015). Failing to establish metric invariance may indicate that although the same media sources are relevant in people’s overall trust calculus across cases, the relations between the different factors are conditioned by context and therefore are incomparable across cases. In contrast, metric invariance would indicate that a one-unit change in trust in one country will be comparable to a one-unit change in all others. Establishing this type of invariance would suggest that the construct has the same meaning to participants across groups, and it would indicate that cross-cultural comparisons of covariances and unstandardized regression coefficients are possible (Bollen 1989; Byrne et al. 1989).
The last step is testing for scalar invariance (or ‘strong invariance’), which requires that item intercepts will also be equivalent across administrations. Scalar invariance allows mean comparisons across groups and when obtained, cross-group differences in the means of the observed items are interpreted as a result of differences in the means of their corresponding latent constructs (Meredith 1993). Scalar invariance is a quality that can be approximated at best (Marsh et al. 2018), as it is unattainable in many cases. In such cases, analysts are satisfied with partial scalar invariance, if at least two indicators have invariant intercepts across all groups (Brown 2015; Byrne et al. 1989). Ascertaining scalar invariance allows substantiating multi-group comparisons of factor means (e.g., T-tests or ANOVA), and provides confidence that any statistically significant differences in group means are not due to differences in scale properties in different groups (Bollen 1989; Jöreskog 1971; Vandenberg and Lance 2000).
Case selection and data
Establishing the validity of any generalized measure of trust in the news media requires testing its equivalence for citizens living in societies with sufficiently different media landscapes, that are nonetheless culturally and politically close enough to make such comparisons valid. Moreover, any cross-case comparison of trust in the news media will only be meaningful if it pertains to cases with similar media landscapes, consumed under similar political, social and economic circumstances. The European Union (EU) is an ideal collection of cases to test the feasibility of measuring trust in the news media invariantly, thanks to the different historical, social and political circumstances of its Member States on the one hand, and their shared political institutions and commitment to a set of agreed upon values and principles on the other.
The data used in the analyses were collected in the framework of the Flash Eurobarometer study 464 (European Commission 2018). This timeframe was chosen to make sure that respondents’ attitudes were not affected by the unusual Covid-19 circumstances. The survey included data collected in all 28 EU Member States (including the UK, as data were collected prior to the completion of Brexit). Data about the attitudes and preferences of at least 1000 citizens were collected in all EU member states except Cyprus, Luxembourg and Malta, where the sample was smaller and included roughly 500 respondents per member state. In total, some 26,576 respondents making a representative sample of the EU population aged 15 or over were interviewed. This study included a battery of questions about trust in different media sources, asking ‘How much do you trust or not the news and information you access through: (1) Printed newspapers and news magazines, (2) Online newspapers and news magazines, (3) Online social networks and messaging apps, (4) Television, (5) Radio and (6) Video hosting websites and podcasts’. Four-point Likert scales, ranging from ‘do not trust at all’ to ‘totally trust’ were used to gauge the level of trust in each medium, with an additional ‘DK/NA’ category for respondents unable or unwilling to answer.
Table 1 shows the summary statistics for the study variables. One clear finding is that different media enjoyed varying levels of trust. Legacy media, with radio first and foremost, followed by television and newspapers were most trusted by respondents. The median score was 3 for news-producing media and 2 for video hosts and social networks. Lastly, item response rates were rather low. While tolerable with legacy media, for online platforms nonresponse rates were 27–34%. Listwise deletion would lead to losing over half of the observations, and imputations were impossible due to the ordinal-level of the original survey items. Since data are missing at random, I employed pairwise deletion in the analyses, which resulted in analyzing information from 26,380 respondents.
Competing measurement models for trust in news media
Path diagrams with three hypothesized trust in news media measurement models are presented in Fig. 1. In the first model, trust in all six media types is hypothesized to be part of a single latent construct, namely news media trust. The rationale behind this model is that every citizen is exposed to a unique combination of stories, sources and media. Some of these, such as radio news, are considered by many as highly reliable, while others, such as social media apps, are seen by many as noncredible (Kiousis 2001; Newman et al. 2018). The attitudes citizens have toward their news media is based on this rich input as well as on their personal dispositions and experience. This combined input is at the basis of their trust calculus, which determines whether or not the media is generally carrying out its social role adequately.
The next two models propose that media trust is driven by two constructs. In model 2, the split is between legacy media and online media. The rationale is that while the public is exposed to various media, there is a qualitative difference in the news reports offered by each of them (Kiousis 2001; Ladd 2012). This distinction is evident considering the consistent differences in levels of trust expressed toward each. While legacy media enjoy higher levels of trust, online news is met with much skepticism (Fletcher and Park 2017; Kalogeropoulos et al. 2019; Tsfati and Cappella 2003). Model 3 slightly modifies model 2, by distinguishing online newspapers from other online, digital-born media, and grouping it together with legacy media in a group that creates news, as opposed to the digital-born media, who typically distribute news created by other media or generated by users. This is justified in light of the differences in median trust scores reported in Table 1, which shows that digital-born media enjoy lower levels of trust compared to all others.
These measurement models were fitted using a multigroup confirmatory factor analysis, where the vector of the observed trust variables \(y\) was assumed to be caused by a vector of latent variables \(\xi\), where \(y\) contains \({y}_{i} (i=1,...,n)\) random variables and \(\xi\) contains \({\xi }_{j} (j=1,...,m)\) common factors. The causal relationship between an observed variable \({y}_{i}\) and latent construct \({\xi }_{j}\) is represented in the regression equation:
where \({y}_{i}\) is the observed measure of trust, \({\xi }_{j}\) is the underlying latent factor causing the observed trust item \({y}_{i}\), \({\lambda }_{i}\) is a factor loading linking \({y}_{i}\) and \({\xi }_{j}\), \({\delta }_{i}\) is the residual variance in \({y}_{i}\) that is unexplained by \({\xi }_{j}\), and \({\tau }_{i}\) is the item intercept for \({y}_{i}\) (i.e., its value when \({\xi }_{j}=0\)). This model implies that \({y}_{i}\) is a manifest indicator of \({\xi }_{j}\). Assuming that \(E\left({y}_{j}\right)={k}_{j}\), \(E\left({\xi }_{j},{\delta }_{i}\right)=0\), and that \(E\left({\delta }_{i}\right)=0\), the expected value of \({y}_{i}\) can be expressed as:
The model implied covariance matrix of the \(y\) observed variable caused by the latent variable \(\xi\) is:
where \(\Sigma\) is a \(p\times p\) variance–covariance matrix among the \(y\) observed variables, \(\Lambda\) is a \(p\times m\) matrix with items’ factor loading \({\lambda }_{i}\) on factor \({\xi }_{j},\) where \(m\) is the number of common factors, \(\Phi\) is an \(m\times m\) variance–covariance matrix among the common factors in \(\xi\), and \(\Theta\) is a \(p\times m\) diagonal matrix containing the residual variance \({\delta }_{i}\).
Trust in news media as a unidimensional index
Invariance in each model was tested using the Maximum Likelihood estimator. For each baseline model, modification indices and expected parameter changes (EPC) were used and constraints were relaxed by introducing covariances between trust items, unrelated to the latent construct. Factor loadings greater than 0.3 were considered satisfying and the goodness-of-fit of each model was calculated using global fit statistics (Brown 2015). These included the Chi square statistic, the comparative fit index (CFI) and the Tucker–Lewis index (TLI). Additionally, two absolute-fit indices were obtained, including the root mean square error of approximation (RMSEA) and the standardized root mean square residual (SRMR). For an acceptable model fit, CFI and TLI must be greater than 0.95 (Bentler 1990; Bollen 1989), RMSE must be below 0.08 (Hu and Bentler 1998, 1999) and SRMR must be below 0.05 (Jöreskog and Sörbom 1996). Lastly, models will be considered unsatisfactory if their CFI drops by over 0.01 between steps, RMSE grows by more than 0.015 or SRMS grows by more than 0.01 (Chen 2007).
In model 1, trust in all six media, legacy and online, was loaded on a single, unidimensional factor. As shown in Table 2, the baseline model was found unsatisfactory. While CFI and SRMR were acceptable, the other two indices did not reach the acceptable thresholds. Following the modification indices analysis, five covariance terms were added to the model, accounting for dependencies between radio and television, radio and print newspapers, television and print newspapers, television and online newspapers and video/podcast hosts and social network apps. The modified model was satisfactory, with CFI and TLI reaching almost 1 and absolute measures falling well below the cut-off points. Next, cross-case configural invariance was tested, to establish that the factor loading structures are identical across cases. As indicated by the fit measures, this model was satisfactory, with all fit indices surpassing the required threshold. In substantive terms, establishing the configural invariance of this model means that all six items were related to the latent factors and they loaded to the same single factor across all cases.
Moving beyond the configural structure of the data, metric invariance was estimated next. Forcing equivalence on the factor loadings across cases was found justifiable, with acceptable fit measures and mostly acceptable changes in them, compared to the configural model. In one of them, namely SRMS, the estimate change was − 0.02, twice the acceptable threshold of − 0.01. However, given that the estimates for all fit indices were in the acceptable range, all other change metrics were also within the acceptable magnitude, and all standardized factor loadings exceeded 0.3, it is reasonable to interpret these findings as indicating that this model is metrically invariant. In substantive terms, this finding indicates that not only are all items related to the latent factor, but that the strength of these relations is consistent across all administrations.
Testing for scalar invariance returned unsatisfactory results. Both incremental fit indices fell below 0.9 and both absolute indices exceeded their acceptable cut-off points. This means that in its basic form, with all intercepts constrained, the model is too restrictive and does not allow cross-case comparisons of the means. This is because there is little certainty about whether differences between populations reflect different levels of trust or rather indicate that some populations responded to the measures differently than others. Relaxing some constraints by allowing some intercepts to vary helped establish partial-scalar invariance. When the intercepts for trust in television, radio, and both print and online newspapers were released, all four goodness-of-fit measures were well over their thresholds. Compared to the metric invariance step, changes in fit measures were moderate and all with acceptable magnitudes. Examining the country residual correlation matrices corroborated the good fit, with no residual larger than 0.1.
The model solution, corresponding with the hypothesized model in Eq. 1, is presented in the matrix below, where the latent structure \({\xi }_{1}\) is the unidimensional news media trust, \({\tau }_{1-4}\) are the non-constrained intercepts, \({\delta }_{i}\) are the error terms and the resulting y’s labels correspond with television, radio, print newspapers, online newspapers, video and podcast hosts and social network applications.
These results mean that the unidimensional measure of trust in the news media is comparable across contexts. Researchers using this measure can assume that it validly gauges people’s trust in the news media, and that cross-group mean differences reflect actual differences in latent factor means. A standardized, unidimensional index was created to validly reflect EU citizens’ trust in their news media. This index is fully comparable across all 28 member states and provides a unique opportunity for researchers interested in incorporating trust in the news media in their cross-cultural analyses, either as a dependent variable for researchers motivated by explaining what makes some groups in the population more trusting compared to others, or as an explanatory factor, to be used in models predicting another social or political outcome.
Figure 2 shows the cross-case locality, spread and skewness of the index, which ranged between − 1.22 and 1.44. The grand mean was 0.1 (with a standard deviation of 0.43 and a grand median of 0.14). The probability distribution was slightly skewed to the left (− 0.43), indicating that EU citizens’ trust in the news media is positive overall, as also reflected in the overall median per-country, which was positive in 23 out of the 28 cases. The kurtosis was 0.05, suggesting that the deviations from the mean were not extreme. While these results are based on cross-culturally equivalent measures, significant variations across countries is observed. This indicates that context is likely to play an important role in people’s trust in the news media, and modeling trust must account for cross-case differences. Testing for intra-cluster correlation indicated that roughly 30% of the variance in this index is attributed to the context respondents experience, likely their cultural, political and economic environment, as well as news media supply side differences across cases.
Three further findings are presented in Fig. 2. First, the news media enjoyed the trust of 75% or more of the public in Croatia, the Czech Republic, Denmark, Estonia, Finland, Latvia, Lithuania, Malta, the Netherlands, Portugal and Slovakia. In the remaining 17 countries, 25% of the population or more expressed negative attitudes, while 25% or more indicated that they trust the media. In contrast, the majority in the Austrian, French, German, Italian and Spanish populations did not trust the news media, and in 13 other countries, namely Hungary, Bulgaria, Luxembourg, Sweden, Greece, Ireland, the UK, Finland, Belgium, Slovenia, Cyprus, Poland and Romania, 25–50% of the population had negative attitudes to the media. Lastly, relative homogeneity of attitudes toward the media was observed in Estonia, Malta, Denmark, the Netherlands and Finland, while in most other countries there was much more variance. Outliers were observed in countries with relatively homogenous distributions.
Trust in news media as a bidimensional index
The next two models tested whether a measure of trust in the news media would better fit a bifactorial structure. In model 2, trust is partitioned between two hypothesized factors: legacy and new/online media. The results from this model are presented in Table 3. We can note that all but one fit measure in the initial overall fit model were unacceptable and required modifications. Specifically, covariation terms were added between radio and television, print and online newspapers, and video hosts and social media apps. This model proved configural and metric equivalence, with an unsatisfying change in SRMS between the two steps, similar to what was observed in model 1. None of the fit measures yielded satisfying results testing scalar invariance and multiple efforts to obtain partial-scalar invariance have failed. The best result was obtained after relaxing the intercept restrictions on television and online newspapers, but the change in incremental fit measures compared to the metric invariance model was too large to accept the results as indicating equivalence. In sum, the bidimensional partition into legacy and online media yielded a metric invariance but not a (partial-) scalar one.
Lastly, the bidimensional model partitioning trust in news media between news-producing and web 2.0 media obtained similar results. As shown in Table 4, the initial overall model fit measures were unsatisfying, and covariance terms between trust in radio and television, and video/podcasts hosts and social network apps were introduced. All fit indices were satisfactory in the modified overall model and remained so in the next one, indicating configural invariance. The changes in three fit indices between the configural and metric model were acceptable, while the change in RMSEA was slightly higher than acceptable (0.02). However, given the strong fit identified by the other indices, we can reasonably conclude that this partition into two factors yields metric invariance. The scalar model returned mixed results, with only the absolute fit indices reaching the threshold. Changes in these indices were beyond the acceptable cutoff point, which made the scalar model invariant. Relaxing some of the intercept constraints did not help in establishing partial scalar invariance, as although all fit measures reached the required threshold, their values changed substantially from the metric model. Thus, we can conclude that similar to the previous bidimensional model, while metric invariance was found, scalar invariance was not.
Discussion and conclusions
This study set out to answer to what extent can the measurement invariance of trust in the news media be established for individuals living in 28, as of 2018, EU Member States. It makes a contribution to the literature on media trust, and specifically to the debates around the conceptualization and measurement of this construct. While this literature often employs various measures of trust in comparative analyses, these measures are often not validated and the scarce attempts to validate them have led to unsatisfying results, providing little ground for comparative analyses. Conceptual and measurement debates still exist in the literature, despite century-long efforts to resolve them. The issue becomes even more important with the changes in the media landscape and the rise of public sentiments critical of social institutions in general and the news media in particular. This sense of crisis is not limited to trust in the media and extends to trust in multiple social and political institutions (Bogaerts and Carpentier 2013; Coleman 2012).
The analysis employed salient conceptualization and used multi-group confirmatory factor analysis to test whether or not three competing measurement models are comparable across different EU Member States. The tests aimed at establishing equivalence in these models’ factorial structure, loadings and intercepts. The findings showed the pertinence in conceptualizing and measuring news media trust using a unidimensional approach, where trust in various media is assumed to be caused by the same latent construct. These findings suggest that it would be more appropriate to understand trust in the news media as an assessment people make of the entire corpus of the news media they are exposed to. While some news media sources enjoy higher levels of public trust, no evidence was found to suggest that people establish their trust in news sources they consume while differentiating between them based on their reputation.
The findings are also encouraging with respect to the potential of large cross-national research on the correlates of trust in news media. Establishing the unidimensional partial scalar invariance of the measurement, suggests that the factors loading structure, the loading values and the values of two of the intercepts do not systematically depend on irrelevant aspects of data collection such as sampling, translation issues, different interpretation of questions, or cultural difference in social desirability and acquiescence. Rather, the levels of trust in news media observed using this index are casually related to the latent scores and can be incorporated into a factor with identical structure, loadings and partially identical intercepts to accurately reflect the real level of trust EU citizens have in their news media.
These findings imply that despite the clear qualitative differences between different media, the level of trust the public has in the news media is based on a calculus that accounts for all media sources combined. When survey respondents are asked about their trust in the media, their answers are not based on the net, individual qualities of each medium, reporter or story. Rather, their trust is informed by the general trustworthiness of the news media they are exposed to. While many respondents may be able to consider media micro-level performance, comparing their levels of trust across cases requires factoring their trust in different media using a unidimensional index. Additionally, while trust in web 2.0 media proved to be constrainable across cases, the cross-case variance between different legacy media was more prominent and cannot be restricted. While this result does not condition the relationship between the latent construct and these four items, it will affect the cross-case variation in mean levels of trust.
The results also indicate that the bidimensional measurement models were metrically invariant, but their scalar invariance, or part of it, was not attainable. The invariance found in these models indicates that people in different EU countries responded to the items in the same way, so that the strength of relations between specific scale items and their respective latent constructs are the same across context. This means that the ranking of different media according to the trust levels they enjoy in the population, as often done by the Eurobarometer, is comparable across countries, and observed item differences will indicate country-differences in the underlying construct. Grouping these items into two dimensions, however, does not constitute a valid measurement model and will lead to biased estimates. Bifactorial measures of trust in the news media must not be employed when testing theories about the sources or consequences of media trust and its cross-context variance.
In this rapidly changing media scene, it is easy to understand why many see the differences between legacy media and online media as two different things. The professionalism, accuracy and fair attitude of legacy media are incomparable to the eclectic and sometimes chaotic nature of online news, and with these differences one may expect that people’s judgement of the media will be differential. A similar argument can be made about the qualitative differences between tabloids and spreadsheets, or between news on a TV morning show and the evening news. However, the modern media environment presents users with hard news and kitten images, all blended into one feed. Many users are aware of these differences, yet they assess the news media as a whole, a summary of all news they are exposed to, pertaining to any story, delivered by any source and disseminated by any medium. This way of judging the media is common to citizens across the EU, despite the grave differences in how the media performs under specific political, economic and social contexts.
This study has two main limitations which provide opportunities for future research. First, the analyses are based on cross-sectional data, which does not allow establishing the invariance of the measures across time, and therefore more research is needed to validate the longitudinal unidimensionality of media trust. Second, these data come from a single area in the world, the EU. While focusing on a collection of countries with cultural and political similarities makes sense for establishing measurement invariance, future research would benefit from extending the selection of cases and testing the model equivalence in additional cases to establish its universality..
The alarming warnings about the decline in media trust and the strong partisan divide that drives them are serious. The term ‘fake news’ is no longer limited to participants in political rallies. It has become part of modern society. The warnings are serious because the trust people have in the media is much more than their mere approval of the quality of reporting. Similar to trust in any other social institution, trust in the news media involves risk. The risk is not limited to the trustor, who may be misinformed and consequently participate in democracy less efficiently. It affects the trustee, the news media itself, which is seen by many as not carrying out its main role of supporting democratic society. While having a critical attitude to social institutions is good for democracy as it allows citizens to communicate their critique and allows institutions to improve, the danger at higher levels and with rapid deterioration is that media critique may transform into media cynicism, with unwanted consequences for modern society and liberal democracy.
Data availability
Flash Eurobarometer 464: Fake News and Disinformation Online (v1.00). (2018). [Data set]. European Commission, Directorate-General for Communication. http://data.europa.eu/88u/dataset/S2183_464_ENG
Code availability
The code used for the analyses is available in the supplementary material.
References
Almond G, Verba S (1963) The civic culture Political attitudes and democracy in five nations. Sage, London.
Appelman A, Sundar SS (2016) Measuring message credibility: construction and validation of an exclusive scale. J Mass Commun Q 93(1):59–79. https://doi.org/10.1177/1077699015606057
Arnett CE, Davidson HH, Lewis HN (1931) Prestige as a factor in attitude change. Sociol Soc Res 16:49–55
Barber B (1983) The logic and limits of trust. Rutgers University Press, New Brunswick
Bauer PC (2021) Clearing the jungle: conceptualising trust and trustworthiness. In: Freitas RBD, Iacono SL (eds) Trust matters: cross-disciplinary essays. Bloomsbury, London, pp 17–34
Bentler PM (1990) Comparative fit indexes in structural models. Psychol Bull 107(2):238–246. https://doi.org/10.1037/0033-2909.107.2.238
Bogaerts J, Carpentier N (2013) The postmodern challenge to journalism: strategies for constructing a trustworthy identify. In: Peters C, Broersma M (eds) Rethinking journalism: trust and participation in a transformed news landscape. Routledge, New York, pp 60–71
Bollen KA (1989) Confirmatory factor analysis. In: Bollen KA (ed) Structural equations with latent variables. Wiley, New York, pp 226–318
Brown TA (2015) Confirmatory factor analysis for applied research. The Guilford Press, New York
Byrne BM, Shavelson RJ, Muthén B (1989) Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychol Bull 105(3):456–466. https://doi.org/10.1037/0033-2909.105.3.456
Carr DJ, Barnidge M, Lee BG, Tsang SJ (2014) Cynics and skeptics: evaluating the credibility of mainstream and citizen journalism. J Mass Commun Q 91(3):452–470. https://doi.org/10.1177/1077699014538828
Chen FF (2007) Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equ Model 14(3):464–504. https://doi.org/10.1080/10705510701301834
Coleman S (2012) Believing the news: from sinking trust to atrophied efficacy. Eur J Commun 27(1):35–45. https://doi.org/10.1177/0267323112438806
Davidov E, Meuleman B, Cieciuch J, Schmidt P, Billiet J (2014) Measurement equivalence in cross-national research. Ann Rev Sociol 40:55–75. https://doi.org/10.1146/annurev-soc-071913-043137
Delhey J, Newton K (2003) Who trusts? The origins of social trust in seven societies. Eur Soc 5(2):93–137. https://doi.org/10.1080/1461669032000072256
Easton D (1975) A re-assessment of the concept of political support. Br J P Sci 5:435–457. https://doi.org/10.1017/S0007123400008309
European Commission, Brussels (2018). Flash Eurobarometer 464 (Fake News and Disinformation Online). GESIS Data Archive, Cologne. https://doi.org/10.4232/1.13019
Exoo CF (2009) The pen and the sword: press, war, and terror in the 21st century. Sage, London
Fisher C (2018) What is meant by ‘trust’ in news media? In: Otto K, Köhler A (eds) Trust in media and journalism: empirical perspectives on ethics, norms, impacts and populism in Europe. Springer, Wiesbaden, pp 19–38
Fletcher R, Park S (2017) The impact of trust in the news media on online news consumption and participation. Digit J 5(10):1281–1299. https://doi.org/10.1080/21670811.2017.1279979
Hanitzsch T, Van Dalen A, Steindl N (2018) Caught in the nexus: a comparative and longitudinal analysis of public trust in the press. Int J Press/poli 23(1):3–23. https://doi.org/10.1177/1940161217740695
Hooghe M (2011) Why there is basically only one form of political trust? Br J Pol Int Relat 13:269–275. https://doi.org/10.1111/j.1467-856X.2010.00447.x
Hovland CI, Janis IL, Kelley HH (1953) Communication and persuasion. Yale University Press, New-Haven
Hu L-t, Bentler PM (1998) Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychol Methods 3(4):424–453. https://doi.org/10.1037/1082-989X.3.4.424
Hu L-t, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 6(1):1–55. https://doi.org/10.1080/10705519909540118
Jöreskog KG (1971) Simultaneous factor analysis in several populations. Psychometrika 36(4):409–426. https://doi.org/10.1007/BF02291366
Jöreskog KG, Sörbom D (1996) LISREL8: user’s reference guide. Scientific Software, Mooresville
Kalogeropoulos A, Suiter J, Udris L, Eisenegger M (2019) News media trust and news consumption: factors related to trust in news in 35 countries. Int J Commun 13:3672–3693. https://doi.org/10.5167/UZH-175863
Kiousis S (2001) Public trust or mistrust? Perceptions of media credibility in the information age. Mass Commun Soc 4(4):381–403. https://doi.org/10.1207/S15327825MCS0404_4
Klingemann HD, Fuchs D (1995) Citizens and the state. Oxford University Press, Oxford
Kohring M, Matthes J (2007) Trust in news media: development and validation of a multidimensional scale. Commun Res 34(2):231–252. https://doi.org/10.1177/0093650206298071
Ladd JM (2012) Why Americans hate the media and why it matters. Princeton University Press, Princeton
Ladd JM, Podkul AR (2019) Sowing distrust of the news media as an electoral strategy. In: Grofman B, Suhay L, Trechsel A (eds) The Oxford handbook of electoral persuasion. Oxford University Press, Oxford, pp 427–454
Marsh HW, Guo J, Parker PD, Nagengast B, Asparouhov T, Muthén B, Dicke T (2018) What to do when scalar invariance fails: the extended alignment method for multi-group factor analysis comparison of latent means across many groups. Psychol Methods 23(3):524–545. https://doi.org/10.1037/met0000113
Mayer RC, Davis JH, Schoorman FD (1995) An integrative model of organizational trust. Acad Manag Rev 20(3):709–734. https://doi.org/10.2307/258792
McCroskey JC, Mehrley RS (1969) The effects of disorganization and nonfluency on attitude change and source credibility. Speech Monogr 36(1):13–21. https://doi.org/10.1080/03637756909375604
Medina TR, Smith SN, Long JS (2009) Measurement models matter: implicit assumptions and cross-national research. Int J Public Opi Res 21:333–361. https://doi.org/10.1093/ijpor/edp037
Meredith W (1993) Measurement invariance, factor analysis, and factorial invariance. Psychometrika 58:525–543. https://doi.org/10.1007/BF02294825
Metzger MJ, Flanagin AJ, Eyal K, Lemus DR, Mccann RM (2003) Credibility for the 21st century: integrating perspectives on source, message, and media credibility in the contemporary media environment. Ann Int Commun Assoc 27(1):293–335. https://doi.org/10.1080/23808985.2003.11679029
Newhagen J, Nass C (1989) Differential criteria for evaluating credibility of newspapers and TV news. J Q 66(2):277–284. https://doi.org/10.1177/107769908906600202
Newman N, Fletcher R, Kalogeropoulos A, Levy DAL, Nielsen RK (2018) Reuters Institute digital news report 2018. Reuters Institute for the Study of Journalism, Oxford
Norris P (1999) Critical citizens: Global support for democratic government. Oxford University Press, Oxford. https://doi.org/10.1093/0198295685.001.0001
Roper BW (1969) A ten-year view of public attitudes toward television and other mass media, 1959–1968. Television Information Office, New York
Self CC, Roberts CR (2019) Credibility. In: Stacks D, Salwen MB, Eichhorn KC (eds) An integrated approach to communication theory and research. Routledge, New York, pp 435–446
Steinmetz H, Schmidt P, Tina-Booh A, Wieczorek S, Schwartz SH (2009) Testing measurement invariance using multigroup CFA: differences between educational groups in human values measurement. Qual Quant 43(4):599–616. https://doi.org/10.1007/s11135-007-9143-x
Strömbäck J, Tsfati Y, Boomgaarden H, Damstra A, Lindgren E, Vliegenthart R, Lindholm T (2020) News media trust and its impact on media use: toward a framework for future research. Ann Int Commun Assoc 44(2):139–156. https://doi.org/10.1080/23808985.2020.1755338
Tsfati Y, Cappella J (2003) Do people watch what they do not trust? Exploring the association between news media skepticism and exposure. Commun Res 30(5):504–529. https://doi.org/10.1177/0093650203253371
Tsfati Y, Cohen J (2005) Democratic consequences of hostile media perceptions the case of Gaza settlers. Harvard Int J Press/pol 10(4):28–51. https://doi.org/10.1177/1081180X05280776
Vandenberg RJ, Lance CE (2000) A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ Res Methods 3(1):4–70. https://doi.org/10.1177/109442810031002
Wu H, Estabrook R (2016) Identification of confirmatory factor analysis models of different levels of invariance for ordered categorical outcomes. Psychometrika 81(4):1014–1045. https://doi.org/10.1007/s11336-016-9506-0
Yamagishi T, Yamagishi M (1994) Trust and commitment in the United States and Japan. Motiv Emot 18(2):129–166. https://doi.org/10.1007/BF02249397
Acknowledgements
The author would like to thank Christine Arnold, Esther Versluis and Regina Lawrence for providing thoughtful feedback on an earlier version of this paper, and to Soetkin Verhaegen and the participants of the Politics and Culture in Europe Colloquium at Maastricht University for their helpful comments and suggestions.
Funding
No funding was received to assist with the preparation of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author has no conflict of interests/competing interests.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Sapir, E.V. Can we trust measures of trust? Measurement invariance in trust in EU news media. SN Soc Sci 2, 226 (2022). https://doi.org/10.1007/s43545-022-00534-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s43545-022-00534-1