Introduction

With the rise of search engines and online databases, traditional approaches to accessing scientific research outputs such as browsing print journals are increasingly being replaced by keyword searches and online browsing (Houghton et al., 2004; Thelwall et al., 2013). In searching for and browsing information from research outputs, readers often use the titles and abstracts of specific research outputs to obtain a preliminary understanding of these outputs and locate materials of interest to them (Pinto & Lancaster, 1999). Widely recognized as “an abbreviated, accurate representation of a document” (Weil, 1970, p. 352), the abstract of a research article (RA) provides a synopsis of the main ideas, research design, and findings of the research that can help accelerate the readers’ grasp of the essential information about the RA and facilitate their judgement of whether it is sufficiently relevant or interesting to warrant access to and reading of the full text (Gazni, 2011). As such, it is not surprising that abstracts tend to be read more frequently than other RA parts (Nicholas et al., 2003).

Meanwhile, thanks to the open access movement of scholarly publishing and the rapid rate of online knowledge dissemination, RAs are now not only referenced by scholars for academic purposes, but also read by an increasingly wide range of non-expert readers seeking information for various purposes (Sud & Thewal, 2014). It is thus more important than ever for RAs to be accessible to readers in both the scientific community and the general public in an era of information overload (Tankó, 2017). Among the factors that may affect readers’ comprehension of RA abstracts, abstract readability has been found to play a fairly prominent role (Dronberger & Kowitz, 1975). At the same time, researchers have reported a lower level of readability of RA abstracts in comparison to full RA texts (King, 1976) and an upward trend of RA abstract difficulty (e.g., Lei & Yan, 2016). In light of the importance of RA abstracts in the scientific reach of RAs, recent research has started to pay attention to the relationship of abstract readability and the scientific impact of RAs (e.g., Didegah & Thelwall, 2013; Gazni, 2011; Sienkiewicz & Altmann, 2016).

Importantly, if RA abstracts prove difficult to read to the scientific community, they may be even more so to non-expert readers (Bornmann, 2014), and abstract readability may have an effect on the reach and impact of RAs among the general public (Lei & Yan, 2016). Exciting efforts to improve research accessibility are emerging. For example, the Open Accessible Summaries in Language Studies (OASIS) initiative “aims to make research into language learning and teaching openly available and easily accessible to anyone who might be interested for professional or other reasons” by asking authors of published RAs to provide a one-page summary in non-technical language (OASIS, 2020). The increasing popularity of the OASIS initiative among language-related academic journals speaks to the recognition of the effect of abstract readability on the dissemination of research outputs to the general public. However, systematic examinations of the relationship between abstract readability and the impact of RAs among general non-expert readers are scant. An understanding of this relationship will provide empirical evidence for the need for such initiatives as OASIS as well as insight into the specific aspects of language use that researchers can attend to in order to improve abstract readability and the social impact of their RAs.

Measuring abstract readability

The readability of a written text is generally understood as the ease with which the text can be read and understood (Klare, 1963). Previous research has shown that the readability of an abstract may be affected by its linguistic features (Crossley et al., 2019) and other factors such as structure and layout (Hartley, 2000; Hartley & Sydes, 1997). Much research on the linguistic aspect of abstract readability has employed traditional readability formulas (Graesser et al., 2011; Klare, 1974), such as the Flesch Reading Ease (FRE; Flesch, 1948) and the Simple Measure of Ginledygook statistics (SMOG; McLaughlin, 1969). Such formulas generally incorporate a measure of word difficulty or familiarity, most commonly number of letters or syllables per word or the proportion of familiar words, and a measure of sentence complexity, most commonly mean sentence length, and generate a single metric that can be used to compare the readability of different texts. Dronberger and Kowitz (1975), for example, observed significant differences between the FRE scores of abstracts and full texts of documents retrieved from the Educational Resources Information Center system and recommended the use of readability formulas to evaluate and modify the difficulty levels of abstracts. Similarly, Lei and Yan (2016) analyzed a corpus of RAs published in four academic journals (i.e., Scientometrics, Journal of Informetrics, Research Policy, and Research Evaluation) from 2003 to 2012 using FRE and SMOG and reported a significant gap between the readability of RA abstracts and full RAs.

Despite the widespread usage of readability formulas for measuring the readability and ease of comprehension of written texts, some scholars have questioned their construct validity because of the simplistic nature of the word and sentence difficulty measures they incorporate (Benjamin, 2012; Crossley et al., 2019; Graesser et al., 2011; Lu et al., 2014). In particular, research in cognitive science has modelled reading comprehension as a coordinated process involving multiple levels of linguistic and discursive complexity, including not only lexical and syntactic complexity but also cohesion and sentiment, among others (Graesser et al., 2011; Just & Carpenter, 1980; Kintsch & van Dijk, 1978). Accordingly, recent research has advocated the use of multiple indices that gauge different dimensions of linguistic complexity in the measurement of the formal linguistic aspect of text readability or difficulty (e.g., Stevens et al., 2015; Tankó, 2017; Vajjala & Meurers, 2012). For example, Vajjala and Meurers (2012) reported that the use of a combination of lexical and syntactic complexity indices outperformed traditional readability formulas by a large margin in predicting the grade levels of reading texts.

Recent studies of abstract readability are also starting to resort to indices of lexical and syntactic complexity. For example, Tankó (2017) employed a set of indices of lexical density, lexical sophistication, lexical variation, and clause length to analyze a corpus of RA abstracts and reported that they “have high syntactic complexity and lexical density and contain primarily low frequency words” (p. 42), which, along with their high information content, makes them difficult to process. Lu et al. (2019), while focusing on whole RAs rather than RA abstracts, showed that indices of lexical complexity (e.g., type-token ratio) and syntactic complexity (e.g., number of clauses per sentence) were useful in revealing the relationship between the readability and scientific impact of RAs.

Abstract readability and scientific impact

The scientific impact of research outputs, often measured quantitatively using citation-based analysis, is an important indicator of the value of the research and of the scholarly success of the researchers producing such outputs (Didegah & Thelwall, 2013; Lu et al., 2019; Sud & Thelwall, 2014). Indeed, RAs that are frequently cited are usually those that have laid an important foundation or made critical discoveries upon which new research can be built (D’Angelo & Di Russo, 2019). The citation counts of research outputs are now readily available from journal websites and/or research databases (Sud & Thelwall, 2014). In addition to the subject matter addressed and the quality of the research presented in RAs, many other factors may affect their scientific impact, such as the impact of the journals they are published in and the references they cite, the scope of collaboration involved in the research, and features pertaining to their writing and organization (Chen et al., 2020; Didegah & Thelwall, 2013).

Among the features examined in the last category, the readability or linguistic complexity of RA abstracts has received substantial attention. This is not necessarily surprising. In order for an RA to be cited by other researchers, it needs to first attract their attention and be read by them. The title and abstract are often the first parts of an RA that are read by researchers looking for relevant research sources and references. In this process, the readability or complexity of the abstract may affect their comprehension of the abstract and potentially their decision to pursue the full text or move on to other sources. Several recent studies have explored the relationship between abstract readability and the scientific impact of research outputs as indicated by their citation counts, although the results reported so far have been somewhat inconsistent. Sienkiewicz and Altmann (2016) reported significant positive correlations between the citation counts of RAs and the complexity of abstracts measured using the Gunning fog index, the z-index, and Herdan’s C measure. Gazni (2011) reported a significant negative linear relationship between the citation counts of published RAs from five prestigious institutions and abstract readability assessed using the Flesch Reading Ease Score. Didegah and Thelwall (2013) considered seven readability formulas and settled on the Flesh Reading Ease Score as a proxy of abstract readability “since it seems to be the most popular and also has a high correlation with the other six scores” (p. 864). However, they reported no significant association between this measure of abstract readability and citation counts of RAs. Notably, most previous studies took discipline and time since publication into account in examining the relationship between abstract readability and scientific impact (D’Angelo & Di Russo, 2019; Htoo & Na, 2017; Maflahi & Thelwall, 2018).

While this line of research has mostly used readability formulas to assess abstract readability, in their analysis of the relationship of whole RA readability to scientific impact, Lu et al. (2019) moved beyond readability formulas but employed a set of lexical and syntactic complexity indices, as mentioned in "Measuring abstract readability" section. Specifically, they reported that high-impact articles tend to have a higher degree of lexical diversity (e.g., a higher type-token ratio), lexical sophistication (e.g., longer nouns), as well as syntactic complexity (e.g., longer sentences) than medium- and low-impact articles.

Online attention and social impact

In recent years, the rise of social media platforms such as Facebook, Google + , and Twitter and the increasing use of such platforms for sharing and discussing research outputs have dramatically improved the visibility of research outputs to the general public (Adie & Roe, 2013; Sugimoto et al., 2017). Conventional measures of scientific impact such as citation counts cannot readily capture the social impact of research outputs among the general public (Mohammadi et al., 2015). For this reason, bibliometrics researchers have started developing measures of social attention received by RAs online (Díaz-Faes et al., 2019). Altmetrics, short for “alternative metrics”, is a term coined to describe web-based metrics of the impact of research outputs using data from online platforms such as social media (Bornmann, 2014). Such metrics are revolutionary in that, with the potential to take into account attention to and activities around research outputs from both research scholars and non-expert readers, they allow for an efficient and comprehensive assessment of the social impact of research outputs. It has been shown that altmetrics can serve as an effective indicator of attention to research outputs in a short time window, as it may take a long time for citations in other publications to appear and be indexed (Thelwall, 2018).

Citation counts and altmetrics usefully complement each other in assessing the scientific and social impact of research outputs, and some research has explored how these two mechanisms may be related to each other. Syamili and Rekha (2017) collected 62 RAs on the Ebola disease and compared their citation counts with several altmetrics variables, such as the number of views, saves, Mendeley shares, and Twitter shares. While they reported significant correlations between citation counts and views, saves, and Mendeley shares, they found that Twitter shares, which best captured attention from the general public, showed a very weak correlation with citation counts. Costas et al. (2015) reported only a weak correlation between the scientific impact and online attention of RAs, and further showed that 55% of the most highly‐cited publications in their database received little attention from the general public. Similarly, Verma and Madhusudhan (2019) offered empirical evidence that highly-cited RAs are not necessarily widely disseminated on social media, while RAs receiving substantial online attention from non-expert readers may have no citation counts at all. All in all, these results support the notion that scientific impact measured using citation counts and social impact measured using altmetrics of online attention are indeed different aspects of RA impact that both deserve research attention.

While the title and abstract of an RA may affect whether researchers retrieve, read, and subsequently cite the RA, they may have a more immediate effect on whether non-expert readers pay attention to and share it online, and the readability of the title and abstract may play an even more important role in this process as non-expert readers generally have less expertise in the subject matter of the RA than expert researchers. While many studies have investigated the effect of abstract readability on the scientific impact of RAs, few studies have examined its effect on the online attention received by RAs. In a study of the virality of RAs, Guerini et al. (2012) examined the relationship between abstract readability, measured using the Gunning fog and Flesch indices, and three metrics of responses to RAs, namely, citations, bookmarks, and downloads. They reported that highly and lowly cited RAs did not differ in abstract readability, and that most bookmarked RAs tended to have less readable abstracts, while the most downloaded RAs tended to have more readable abstracts. The altmetrics of online attention (i.e., bookmarks and downloads) used in their study likely still reflected attention more from the scientific community rather than non-expert readers, many of whom may simply browse through RAs or share them on social media rather than actually bookmark or download them. In a recent study of the relationship between abstract readability and online attention, Ngai and Singh (2020) paid special attention to non-expert readers’ interest in RAs. Specifically, they employed altmetrics data provided by Altmetric.com to measure public attention to RAs on new media and social media and reported a positive correlation between the use of metadiscourse and altmetrics attention score. However, the study focused on a single aspect of abstract readability only, i.e., the facilitative power exerted by metadiscoursal devices to persuade and engage readers, rather than measuring it with multiple indices of formal linguistic features. A better understanding of the relationship between abstract readability and online attention would call for the use of both multidimensional readability models that can comprehensively predict linguistic complexity and integrative altmetrics measurement that can better reflect the wide range and types of ways through which non-expert readers pay attention to and interact with RAs.

Research questions

The goal of this study is to examine the relationship between abstract readability and the online attention received by RAs. Specifically, the study aims to answer the following two research questions:

  1. (1)

    What is the relationship between lexical and syntactic features of abstract readability and the level of online attention received by RAs?

  2. (2)

    How is the relationship affected by publication time and discipline?

Method

The corpus

Our data consisted of a corpus of abstracts of articles published in the Reports column of the Science journal. The journal Science was selected for two reasons. First, it is ranked among the world’s most high-impact outlets of cutting-edge research, has broad disciplinary coverage (with 43 distinct disciplines represented), and enjoys a wide international authorship and readership. Second, as indicated in its mission and scope statement, Science seeks to publish influential articles that merit recognition not only by the scientific community but also by the general public (Science, 2020a), making it especially appropriate for the analytical focus of the current study on online attention to RAs from both the scientific community and the general public. Articles in the Reports column “present important new research results of broad significance” (Science, 2020b), are about 2500 words in length, and are required to contain a standalone abstract of around 125 words that “explain[s] to the general reader why the research was done, what was found and why the results are important” (Science, 2020c).

Data collection proceeded in three steps. We first downloaded all articles published in the Reports column of the Science journal in 2012 and 2018 from its official website (https://science.sciencemag.org/content/collections.) in May, 2019. These two years were selected to examine the potential effect of time on the amount of online attention received by the articles. Next, we filtered disciplines with fewer than 25 articles in either year. A total of 11 disciplines remained, including Biochemistry, Cell Biology, Chemistry, Ecology, Evolution, Geophysics, Immunology, Materials Science, Molecular Biology, Neuroscience, and Physics. Finally, we randomly sampled 25 articles for each of the 11 disciplines for each of the two years. As summarized in Table 1, the final corpus contained 550 abstracts, with a total of 68,859 running words (Mean = 125.20, SD = 13.94).

Table 1 Descriptive statistics of the corpus
Full size table

Readability measurement

Following recent research that advocated for and empirically validated the use of indices reflecting different dimensions of linguistic complexity for readability measurement (e.g., Crossley et al., 2019; Graesser et al., 2011; Lu et al., 2019; Tankó, 2017; Vajjala & Meurers, 2012), we assessed the readability of each abstract in the corpus using a comprehensive set of lexical and syntactic complexity indices.

The Lexical Complexity Analyzer (LCA; Lu, 2012) was employed to measure the lexical complexity of each abstract. LCA contains 25 measures that gauge three dimensions of lexical complexity, namely, lexical density, lexical sophistication, and lexical variation, as summarized in Table 2. Lexical density is calculated as the ratio of the number of lexical words to the total number of words in a text. Lexical sophistication refers to the degree to which the words used in a text are advanced, and, following Laufer and Nation (1995), LCA operationalizes sophisticated words as low frequency words (i.e., words beyond the top 2000 most frequent words in the British National Corpus). This operationalization is both practical and useful, as more frequent words have been found to be processed or decoded faster in reading (Haberlandt & Graesser, 1985; Perfetti, 2007). LCA offers five lexical sophistication measures that each tap into the proportion of sophisticated lexical word tokens (LS1), sophisticated word types (LS2), and sophisticated verb types (VS1, CVS1, and VS2). Lexical variation assesses the degree of diversity of the words used in a text. Texts with greater lexical variation may create more gaps in cohesion and lead to greater difficulty in reading comprehension than texts with greater lexical repetition (Graesser et al., 2011). LCA offers 19 lexical variation measures of three broad categories. The first four are based on the idea of number of different words (NDW). In addition to NDW in the text, three additional measures are included to mitigate the effect of text length on NDW, i.e. NDW in the first 50 words of the text (NDW-50) and the average NDW of 10 random 50-word samples from the text (NDW-ER50) or 10 random 50-word sequences from the text (NDW-ES50). The next six are based on the idea of type-token ratio (TTR), including both the original TTR measure and five transformations of the measure to mitigate the effect of text length (MSTTR-50, CTTR, RTTR, LogTTR, and Uber). The final nine measures gauge the degree of variation of specific types of words, including lexical word variation (LV), verb variation (VV1, SVV1, CVV1, and VV2), noun variation (NV), adjective variation (AdjV), adverb variation (AdvV), and modifier variation (ModV).

Table 2 Lexical complexity measures from Lexical Complexity Analyzer (Lu, 2012)
Full size table

The L2 Syntactic Complexity Analyzer (L2SCA; Lu, 2010) was used to assess the syntactic complexity of each abstract. Based on Wolfe-Quintero et al. (1998)’s recommendation and categorization, L2SCA offers 14 measures that reflect five dimensions of syntactic complexity, including length of production unit, amount of subordination, amount of coordination, degree of phrasal complexity, and overall sentence complexity, as summarized in Table 3.

Table 3 Syntactic complexity measures from L2 Syntactic Complexity Analyzer (Lu, 2010)
Full size table

The LCA and L2SCA are both freely accessible through command-line and web-based interfaces, as detailed in Lu (2014); additionally, the L2SCA is also accessible through a graphic-user interface and as a R package.Footnote 1 The reliability of LCA depends on the reliability of the tagger and lemmatizer used for POS tagging and lemmatization. For example, the web-based LCA uses the Stanford tagger (Toutanova et al., 2003) and the MORPHA lemmatizer (Minnen et al., 2001), which have accuracy levels of over 97% and 99%, respectively. Lu (2010) evaluated the reliability of the L2SCA on a set of English essays written by Chinese college students and reported correlations ranging from 0.834 to 1.000 between the scores calculated by human annotators and those generated by the L2SCA. In terms of their usefulness for text readability analysis, Vajjala and Meurers (2012) reported that using 15 of the 25 measures from the LCA measures (including the LD measure and all lexical variation measures other than the four NDW-based ones) along with an additional measure of lexical variation achieved an accuracy of 68.1% for predicting the grade levels of reading texts, while using the 14 measures from the L2SCA achieved an accuracy of 71.2%.

Attention measurement

The online attention received by each RA in our dataset was quantitatively assessed using the Altmetric attention score (AAS, provided by Altmetric.com) for each RA retrieved from the official website of Science on May 14, 2019. The AAS of an RA has been considered a reliable and convenient reflection of the quantity and quality of the online attention it receives, as it weights the mentions of the article by different authors from different types of sources (Adie & Roe, 2013). In particular, three variables play an important role in computing the AAS of an RA, namely, the volume, authors, and sources of the online mentions of the RA (Trueger et al., 2015). The AAS of an RA goes up with the increase of the volume of its online mentions, i.e. the number of times it is shared or discussed. Meanwhile, the contribution of an author’s mention of the article to its AAS is weighted by the author’s profile, including reach (number of followers), promiscuity (frequency of mentioning research outputs), and bias (frequency of mentioning research outputs from the same journal) (Trueger et al., 2015). Finally, the AAS of an RA is presented in a colored “donut” with different colors representing different types of sources of its mentions, such as news outlets, policy documents, Wikipedia, and social media (Brigham, 2014). This is illustrated by Fig. 1, in which the number at the center of the donut is the AAS received by a certain article, and the various colors represent the proportions of its mentions from different sources. Table 4 shows the descriptive statistics of the AASs for all RAs in our corpus by discipline and publication time.

Fig. 1
figure 1

Diagrammatic representation of the Donut (Altmetric, 2020)

Full size image
Table 4 Descriptive statistics of AASs by discipline and publication time
Full size table

Statistical analysis

We first conducted a two-way ANOVA with publication time and discipline as independent variables and the AASs of the Science articles as dependent variable to determine the effects of the two control variables and their interaction on the online attention received by the articles. To account for the considerable variation across years of publication and disciplines, the raw AASs were normalized using the field-normalization approach (e.g., Huang et al., 2018; Lei & Yan, 2016; Lu et al., 2019). Specifically, we calculated normalized AAS (NAAS) using the following formula:

$${\text{NAAS}} = {\text{article's AAS/mean AAS of the year and discipline in which the article is published}}$$

We then calculated the correlation coefficients between each of the 39 linguistic complexity indices and the NAASs and also examined the correlations between different publication times and among different disciplines. Finally, we constructed a series of multiple regression models with the NAASs as the dependent variable and the linguistic complexity indices as independent variables, using the stepwise method to exclude insignificant predictors (Larson-Hall, 2015).

Results

The two-way ANOVA showed significant effects of publication time (F (1, 528) = 143.98, p < 0.001, ηp2 = 0.214), discipline (F (10, 528) = 14.68, p < 0.001, ηp2 = 0.218), as well as the interaction between the two (F (10, 528) = 8.87, p < 0.001, ηp2 = 0.144) on the AASs of the articles. Tukey’s HSD post hoc tests showed that the AASs of the articles published in 2018 (Mean = 244.35) were significantly higher than those published in 2012 (Mean = 50) (p < 0.001), and the AASs of the articles in Ecology (Mean = 412.09) were significantly higher than those in all other disciplines (p < 0.001). Therefore, to exclude the effect of these two variables, we used NAASs in our subsequent analysis.

Table 5 presents the correlation coefficients between the NAASs of the articles and the 39 linguistic complexity indices computed using the entire dataset. The results showed that overall, three lexical complexity measures (i.e., LS2, NDW-50, and AdvV) significantly correlated with NAASs.

Table 5 Correlations between NAASs and linguistic complexity indices (N = 550)
Full size table

We further examined the correlation coefficients between the linguistic complexity indices and NAASs for articles in each of the 11 disciplines published in each of the two publication times (N = 25 for each of the 22 groups)—the size of the correlation matrix (25 indices, 2 publication times, and 11 disciplines) makes it impractical to present it here. Many more significant correlation coefficients emerged for different groups. Notably, many indices exhibited substantial differences in their correlation coefficients with NAASs across these groups. For example, the correlation coefficient between MLS and NAASs was −0.448 (p < 0.05) for Physics articles published in 2012, 0.174 (p = 0.405) for Physics articles published in 2018, and 0.045 (p = 0.294) for all articles in the dataset.

Given these results, we entered all 39 linguistic complexity indices as predictors of NAASs in constructing multiple regression models, using the stepwise method to exclude insignificant predictors. Tables 6 and 7 summarize the proportion of variance of NAASs explained by each regression model and the coefficients of each regression model, respectively. As these results indicated, the best model (Model 5) included five significant predictors (AdvV, LS2, CN/T, NDW-50, and VP/T) and accounted for 5.3% of the total variance of NAASs. The Tolerance (> 0.10) and VIF (< 10) values of these predictors indicated that multicollinearity was not a concern.

Table 6 The proportions of variance explained by the regression models
Full size table
Table 7 Coefficients of the regression models
Full size table

Discussion

Our analysis of the relationship between abstract readability, measured using 39 lexical and syntactic complexity indices, and the online attention received by 550 RAs from 11 disciplines published in Science in 2012 and 2018, measured using their AASs, revealed a number of substantive findings. First, our analysis confirmed significant effects the two control variables, publication time and discipline, on the online attention received by the RAs. Second, while only three complexity indices correlated significantly with the NAASs of the RAs in the whole dataset, the correlation coefficients between these indices and the NAASs were found to vary substantially by publication time and discipline. Finally, a multiple regression analysis using the 39 complexity indices as predictors generated a model that accounted for 5.3% of the total variance of the NAASs, with five significant predictors. We discuss these results in relation to the two research questions and their implications below.

The relationship between abstract readability and the online attention received by RAs

The results pertaining to our first research question showed that abstract readability significantly affects the online attention received by RAs. The five indices that made their way into the best regression model included one lexical sophistication index (LS2), two lexical variation indices (AdvV and NDW-50), and two phrasal complexity indices (CN/T and VP/T). A few previous studies have examined the relationship between abstract readability and the scientific impact of RAs (e.g., Didegah & Thelwall, 2013; Lei & Yan, 2016). As noted earlier, online attention differs from scientific impact, which is usually measured using citation counts, in that the major contributors of online attention are non-expert readers rather than researchers and the two groups of readers may be sensitive to the readability and complexity of RA abstracts in different ways.

Lexical sophistication negatively affected the online attention received by RAs, as indicated by the significant negative coefficient of LS2. This result suggests that non-expert readers’ comprehension of and attention to RAs may be negatively impacted by the proportion of sophisticated word types, many of which are likely academic and technical words, in RA abstracts. Lu et al. (2019) reported that high-impact RAs used more sophisticated words, suggesting that the use of more sophisticated academic or technical words does not necessarily present a greater comprehension challenge to researchers, but may in fact contribute to a more positive perception of the research and/or writing quality of the RA, making researchers more likely to read the RAs carefully and cite them (e.g., Dolnicar & Chapple, 2015). In other words, the different effects of lexical sophistication on online attention and scientific impact are consistent with the differential level of academic and technical expertise between non-expert readers and expert researchers. The contrast between our results and those reported by Lu et al. (2019) highlights the difference between researchers’ and non-expert readers’ perceptions of RA readability (Newbold & Gillam, 2010), considering that scholarly references are the only source of citations while media outlets (e.g., online news and blogs) are weighted heavily in the calculation of AASs (Altmetric, 2021).

Lexical variation had the greatest effect on online attention. Two lexical variation measures (i.e., AdvV and NDW-50) were positive predictors of NAASs, accounting for nearly half of the total variance explained by all indices in the final regression model. These results suggest that a greater number of different words in the abstract positively impacted online attention. Particularly, a higher degree of adverb variation was useful in attracting more online attention. Although greater lexical variation may create a larger gap in text cohesion and greater processing burden for readers (Graesser et al., 2011), our results show that the richer content encoded by more diverse words may outweigh the processing challenge they pose in sustaining readers’ attention and interest. Lu et al. (2019) reported that high-impact RAs had a higher degree of lexical diversity measured using TTR, suggesting similar attractiveness of lexically diverse RAs to researchers. Taken together, these results indicate that greater lexical variation may contribute to both researchers and non-expert readers’ positive perception of the writing quality of RAs, and subsequently increase their likelihood to read and share the RAs.

Two syntactic complexity indices, i.e. CN/T and VP/T, were included in the final regression model. CN/T was a positive predictor of NAASs, while VP/T was a negative predictor. Complex nominals are commonly used in RAs to achieve informativeness and concision at the same time through the use of modifiers (Lu et al., 2020; Snow, 2010). A higher density of complex nominals in RA abstracts not only makes the abstracts more academic and formal but also allows the authors to pack more information in the limited space allowed. Our results show that these features may have positively affected non-expert readers’ attention to RAs and their positive effect was not outweighed by the additional structural complexity brought about by the greater use of complex nominals. On the contrary, greater use of verbal phrases negatively affected online attention, as indicated by the significant negative coefficient of VP/T. This may be due to non-expert readers’ reliance on linguistic style as a criterion of RA quality. Complex nominals have been shown to be a highly important feature of academic writing (e.g., Lu et al., 2020), and many complex nominals in RAs may have arisen from nominalizations, which are recognized to be “crucial to the conciseness expected in academic language” (Snow, 2010, p.452). It is possible that non-expert readers may judge RAs with the concise and authoritative style realized through complex nominals in general and nominalizations in particular more positively than those employing more clausal structures (both finite and non-finite).

The impact of publication time and discipline

The AASs of the RAs varied significantly by publication time, and the correlations between the complexity indices and AASs varied substantially by publication time as well. We stretched the time span from the first year AASs became available (2012) to the year before our data collection (2018) to maximize the potential effect of time on cumulative online attention. Time generally positively affects citation counts, and we expected a similar effect of time on cumulative online attention. The result that RAs published in 2018 had a higher mean AAS than those published in 2012 was thus initially surprising. Several reasons may help explain this difference. First, while it takes time for RAs to be cited and the scientific impact of RAs tend to grow over time, online shares and mentions of RAs happen instantaneously and tend to concentrate on recent RAs. Second, the past decade witnessed a rapid expansion of the use of the Internet for knowledge discovery and dissemination as well as the use of social media in general, likely leading to increased online reading and sharing by a wider non-expert reader base. The increase in online reading by a wider non-expert reader base likely contributed to greater online attention of RAs published in 2018 than those published in 2012. Meanwhile, as more readers with more diverse levels of expertise participate in online reading and sharing, it is not surprising that the relationship between abstract readability and online attention shifts accordingly.

In addition, the AASs of the RAs and the relationship between abstract readability and online attention varied by discipline as well. The mean AAS of Ecology RAs reached 412.09, significantly higher than the means of RAs in all other disciplines, and noticeably higher than even the second highest mean AAS, which was 234.98 for Evolution RAs. Physics RAs had the lowest mean AAS, which was 54.20. Among the top 100 RAs with the highest AASs in our corpus, more than a quarter (26) were from Ecology, while only one was from Physics. Previous studies also reported substantial between-discipline variation in AASs and Mendeley readership (Htoo & Na, 2017; Zahedi et al., 2014). The disciplinary variation in AASs may to some extent reflect the interest of the general public in research in different disciplines. Non-expert readers’ differential levels of interest and expertise in research in different disciplines likely also affected the role of abstract readability in the online attention received by RAs from different disciplines.

Implications

In an era of mass online reading and sharing facilitated by digitalization and social media platforms, the impact of research outputs is increasingly extending from within the research community to outside the academia. The differences in disciplinary knowledge expertise and reading motives between researchers and non-expert readers will result in differences in their comprehension of and interest in research outputs (Coiro, 2021). As the part of research outputs read most frequently by online readers (Nicholas et al., 2003), abstracts play an especially important role in expanding the reach of RAs among online readers, given the vast information available online that competes for their attention and their usually short attention span for any particular information source. Our results highlight the importance to make RA abstracts more readable and accessible to non-expert readers in order to increase the impact of research outputs among the general public and facilitate knowledge dissemination in the digital era. The results from our regression analysis also offer several specific lexical and syntactic complexity features that authors can pay attention to in an effort to make RA abstracts less challenging to and more appealing to non-expert readers.

Our results also have useful methodological implications for further research in this area. First, while traditional readability formulas merge lexical and syntactic difficulty into a single metric, our results indicate that it is useful to differentiate lexical and syntactic complexity measures in assessing abstract readability both to better understand their respective effect on online attention and to pinpoint the specific features that researchers can focus on in their efforts to improve abstract readability and increase online attention. Second, while traditional readability formulas usually use mean length of sentence as the measure of syntactic difficulty, our analysis using 14 indices reflecting multiple different dimensions of syntactic complexity indicated the importance of considering phrasal complexity in this line of research. Third, our results also showed that publication time and discipline should both be taken into account in future research on online attention. Discipline-specific analysis of the relationship between abstract readability and online attention of RAs will provide more relevant insight for researchers in particular disciplines.

Conclusion

This study examined the relationship between abstract readability, measured using 39 indices of lexical and syntactic complexity, and the online attention received by RAs, measured using Altmetric attention scores, with a corpus of 550 RAs from 11 disciplines published in 2012 and 2018 in the Reports column of Science. Our analysis revealed a number of lexical sophistication, lexical variation, and phrasal complexity measures that significantly predicted the amount of online attention received by the RAs. The analysis further showed that both publication time and discipline may affect the online attention received by the RAs as well as the relationship of abstract readability to online attention. Our findings provide empirical evidence for the importance of making RA abstracts accessible to non-expert readers in order to increase the impact of research outputs among the general public. Our findings also offer several specific linguistic complexity features that researchers can tap into to increase the readability of RA abstracts and maximize reader comprehension and interest.

Several limitations of the current study and avenues for future research exist. First, the scale and scope of our dataset can be expanded to include a greater number of RAs from more diverse journals, disciplines, and publication periods. Second, emerging initiatives to make research output accessible, such as the OASIS initiative, are highly plausible, and it will be revealing to assess non-expert readers’ perception of the readability differences between original RA abstracts and the accessible summaries as well as the effect of those differences on their understanding of and interest in the research outputs and their likelihood to share the research outputs. Third, in addition to lexical and syntactic features, future research may include additional readability indices, such as those of cohesion and sentiment analysis (e.g., Crossley et al., 2019), to examine abstract readability in a more comprehensive way and better understand its effect on online attention. Finally, abstract readability is only one of many factors that can influence non-expert readers’ interest in and attention to research outputs, and it will be useful to examine the interaction between abstract readability and such factors as research topic, title readability, number and complexity of keywords, researcher or institution prestige, and journal impact, among others.