Introduction

As many researchers have noted, social media tools like Twitter, Facebook, and YouTube provide social platforms for individuals to congregate and engage in formal and informal learning opportunities, sharing, and connecting with one another (Greenhow 2011). Recently, researchers have proposed that such tools might also help address some of the most problematic features of Massive Open Online Courses (MOOCs) and engender greater engagement and social support (e.g., Zheng et al. 2016). Yet such claims are supported by little empirical evidence. The existing research exploring these potential benefits has been conducted with individual courses and convenience samples, making it difficult to know to what extent research results are generalizable. Although social media participation and interactions may indeed benefit MOOC participants, further and larger-scale research is needed to yield a more general understanding of who participates on social media and in what ways they do so.

This study examines social media use in MOOCs by taking advantage of the fact that one particular MOOC provider/platform generates hashtags for all of its courses, and therefore provides a naturalistic setting for this research. Because hashtags allow users to find others posting information on similar topics and to share information in an organized fashion, they afford an opportunity to examine how social media, and Twitter in particular, is used across many MOOCs.

Examining all the hashtags generated for all courses offered in 2015 by this MOOC provider, I was able to identify 116 unique courses for inclusion in this study. Next, I used the Twitter API to retrieve the tweets tagged with these hashtags, information on the users who posted the tweets, and relevant metadata associated with the tweets and users. The retrieved data were then analyzed using descriptive and qualitative analysis techniques. Following a review of the literature relevant to this topic, I describe the methods used in this investigation, present and discuss my findings, and conclude with suggestions for future research.

Review of relevant literature

As noted by previous researchers, the contemporary Web is a “read-and-write” platform that enables learners and instructors to engage in participatory practices and to contribute, consume, share, and remix content (Greenhow et al. 2009). Users are typically able to enact these practices on social media—technologies that allow for content consumption as well as user contributed-content—relatively effortlessly. Education researchers have generally been hopeful about the positive impact of social media on education. For instance, they have hoped that social media could provide opportunities for learning that may be richer than those traditionally provided by Learning Management Systems (Brady et al. 2010; Wang et al. 2012), and may foster collaboration, engagement, interaction, knowledge- and resource-sharing, and peer-support (e.g., Ajjan and Hartshorne 2008; Cheston et al. 2013; McLoughlin and Lee 2007; Prestridge 2014). Such hopes however, have rarely been realized and the educational benefits of social media use in education are contentious (Manca and Ranieri 2013; Selwyn 2009; Selwyn and Stirling 2016).

As in the literature on social media in education more generally, researchers examining the use of social media in MOOCs have argued that social media can engender positive outcomes similar to the ones described above. However, only limited research has been conducted on MOOC users’ social media participation. Among those studies, Salmon (2015) found that learners reported using social media for networking purposes; Alario-Hoyos et al. (2014) discovered that participants preferred social tools within the MOOC platform than those outside of it, such as Facebook and Twitter; Knox (2014) reported significant activity on social media in relation to the centralized MOOC platform used in the course; and Zheng et al.’s (2016) study of three MOOCs found that participants stayed longer and participated more on those courses’ official Facebook page than on the course forums. According to the participants interviewed by Zheng et al. (2016), Facebook afforded a sense of community, provided a better and more convenient location for interactions, and made it easier than the course forums to interact with instructors. Although researchers have noted that some learners may object to the use of social media (e.g., Kop et al. 2011; Salmon et al. 2015) and that only a small number of users are responsible for social media activity observed (e.g., van Treeck and Ebner 2013), the ways that social media are used in MOOCs may vary as the design of particular MOOCs varies (cf. Conole 2013). Overall though, the general consensus among researchers in this area is that social media can enhance learning experiences and complement MOOC discussion boards in significant ways.

Twitter is one social media platform that features prominently in the current literature investigating the use of social media in MOOCs. Twitter is a free microblogging platform that allows users to post content in the form of “tweets” that may also contain links to online content. Tweets are limited to 140 characters of text and may be hashtagged with keywords (e.g., #econ1510) or mention other users by username (e.g., @ProfJane). As noted above, some MOOCs include hashtags as a way to provide a social space for learners to congregate. Hashtags are frequently used by researchers to gather data associated with MOOCs (e.g., García-Peñalvo et al. 2015; Koutropoulos et al. 2014; Skrypnyk et al. 2016).

A handful of studies have examined posts on Twitter hashtags in relation to MOOCs. For instance, Alario-Hoyos et al. (2014) noted that learners used Twitter to respond to instructor questions, post resources, and share quotes from video lectures; Koutropoulos et al. (2014) identified ten categories of tweets posted by participants: resource or news item, commentary on participation, reflection on learning, tweets that expressed excitement, calls for help, casual conversation, tweets that expressed thanks, tweets that publicized the MOOC, humorous or entertaining tweets, and tweets that were unrelated to the MOOC. Liu et al. (2016) reported similar findings. In their study, these researchers analyzed the content of social media posts and noted that posts reflected an environment “to share resources, connect with others, enhance communications, and post personal feelings or reflections of learning in an informal and quick manner” (Liu et al. 2016, p. 22).

Although the existing literature begins to paint an initial picture of social media use in MOOCs in general and Twitter in particular, all of the research in this area consists of case studies of one or at most three MOOCs. Although suggestive, these studies pose three significant challenges. First, it is unclear whether or to what extent these findings can be generalized to MOOCs beyond those studied. Second, nearly all of the studies investigating social media use in MOOC contexts examine a short period of time, reflecting the broader research examining microblogging in education (Gao et al. 2012). Finally, large-scale studies examining who chooses to participate on MOOC hashtags, the types of messages they post there, and how those messages vary by user type have rarely been conducted. The research study reported here addressed these limitations by studying hashtags used in numerous MOOCs over a 1-year period, which to my knowledge is the first study to examine as many hashtags over an extended period of time.

Methods

This study examines Twitter as a social media platform used by a large-scale sample of MOOC participants, using a combination of data mining, qualitative, and quantitative methods to answer the following research questions:

RQ 1.:

What general participation patterns can be found in users’ participation on MOOC hashtags?

RQ 2.:

What kinds of users contribute the most to these hashtags and how does participation vary by their role?

RQ 3.:

What is the content of the tweets tagged with these hashtags and how does it vary by users’ role?

Sampling

ManyMOOCs is a pseudonym used to refer to a MOOC provider and platform used by educational organizations to offer courses. Each course offered on this platform has its own unique hashtag. ManyMOOCs provided me with a list of all of its courses offered in 2015 and their published hashtags. An examination of these data revealed that some courses were offered more than once during 2015. Courses offered more than once were eliminated from the dataset to avoid potentially skewing its results, resulting in a final set of 116 courses.

Data collection

The study used a web script developed in past studies (e.g., Kimmons and Veletsianos 2016; Veletsianos and Kimmons 2016) to gather all the unique tweet identification numbers and user identification numbers for each retrievable tweet that included any of the 116 hashtags. The tweet and user identification numbers were then used with a series of custom PHP/MySQL scripts to methodically query the Twitter (2016) Application Programming Interface (API) and extract additional tweet and user information. All tweets and user profiles included in this study were publicly available on Twitter.

The data retrieved by the Twitter API related to tweets and users. The tweet data consisted of each tweet’s unique identifier, creation date, tweet text, userID, and course hashtag. The data retrieved about each user who posted a tweet containing one of the 116 identified hashtags consisted of the user’s userID, screen name, location, and bio description. Both sets of data were stored in a relational database for reconstruction and analysis.

Data analysis

The mode of analysis varied for each of the research questions. RQ1, RQ2, and RQ3 required descriptive analysis. For this, results were generated directly from the database (e.g., queries, string searches) or in Microsoft Excel (e.g., summary statistics). RQ2 and RQ3 also required thematic analysis methods. To answer RQ2, from the database I extracted the users who contributed ten or more tweets into a spreadsheet. To answer RQ3, I exported a representative random sample of tweets from the database. For RQ2, two researchers analyzed user bios and tweets, and coded them to create user categories. For RQ3, two researchers analyzed the text of the tweets, and coded them to generate categories describing the types of tweets posted by participants. To reduce the incidence of bias in the qualitative portion of the research, two researchers conducted analyses independently and then discussed and resolved disagreements. Further details on the analysis of data as it pertains to each research question are provided below.

Limitations

Among the limitations of this study is that some social media participation was not discernible from the public data retrieved for use in the study. Such data include tweets that do not include official hashtags, tweets that are not posted publicly (such as direct or other private messages between users), or tweets that are not returned by the Twitter API due to restrictions imposed on its use. Calculations show that about 80% of all posted tweets that included an official hashtag were retrieved for this study. Further, the methodology employed does not allow the identification of users who make use of these hashtags but do not post messages to them, such as users who read but do not post tweets. As a result, it is possible that the study’s results may underestimate activity pertaining to these courses on Twitter overall. In addition, this study examined only Twitter use and thus the findings may not extend to other social media platforms, such as Facebook.

Results

A total of 16,423 tweets generated by 4931 unique accounts were posted to the 116 hashtags included in this study. The results of the analyses described above are organized according to each research question. Even though enrolment numbers for these courses are unavailable, the number of unique accounts posting tweets is small compared to the numbers or participants MOOCs have historically attracted.

RQ 1: what general participation patterns can be found in users’ participation on MOOC hashtags?

The results indicate that most of the users contributed posts to a single course and that their participation on MOOC hashtags was sporadic and unevenly distributed. The median user in this data set contributed one tweet to one course hashtag. On average, users contributed 3.31 tweets (SD = 13.91; min = 1; max = 614) to 1.17 hashtags (SD = 1.65; min = 1; max = 99). The overwhelming majority of users (92.72%) contributed to just one course; very few (7.28%) contributed to more than one course, and even fewer (1.76%) contributed to more than two courses.

More than half of the users (63.76%) contributed only one tweet, and almost all (94.83%) contributed nine or fewer tweets. Of the 4931 accounts examined, just 255 contributed 10 or more tweets each; these twitter accounts contributed a total of 7985 tweets, or 48.62% of the total tweets in the sample. As shown in Fig. 1, the number of tweets decays exponentially across users. In summary, while many users posted between 1 and 9 tweets, the number of users who posted more than 10 tweets diminished dramatically.

Fig. 1
figure 1

Numbers of users posting tweets

Full size image

Descriptive statistics of users mentioned in tweets showed some evidence of socialization. More than half of the harvested tweets (56.97%) mentioned other users. However, evidence of interaction was weaker, as only about 15% of the posted tweets were replies to others. More than half of all tweets (64.78%) included a hyperlink, indicating that participation on Twitter tends to reference other resources on the Internet and may be outward-looking.

RQ 2: what kinds of users contribute the most to these hashtags and how does participation vary by their role?

Next, I attempted to determine what kind of users contributed the most to these hashtags, identifying those who were ongoing or persistent contributors and determining whether those individuals were learners in the courses or affiliated to the courses in some other capacity. To answer this question, two researchers categorized the 255 users who contributed 10 or more tweets into groups according to their identifiable roles. First, one researcher read 30 user bios and as many tweets posted by each of those users as necessary to categorize them, resulting in four categories. Next, a second researcher used those four categories to code all 255 users using the same method. At the end of this process, the researchers discussed the categories and their fit with the data and determined that two categories required refinement and two additional categories were necessary to describe all accounts. Next, the researchers investigated each assigned code until they reached consensus on the categories into which each individual user belonged. The final categories and their numbers are as follows:

  • Learners. Almost half (114, or 45%) of the individuals in the sample comprised of people who were identified as participating learners. On average, these learners contributed 19.9 tweets (SD = 6.6; min = 10; max = 231) to 1.88 hashtags (SD = 1.62; min = 1; max = 11).

  • Instructors. About one-third (76, or 30%) of participants comprised of individuals identified as having instructional roles in the studied MOOCs. This category included lecturers, facilitators, mentors, and invited speakers that performed facilitative and instructional activities through their tweets. On average, these instructors contributed 33.8 tweets (SD = 46.9; min = 10; max = 369) to 1.42 hashtags (SD = 0.85; min = 1; max = 5).

  • Institutions. Approximately one-sixth (42, or 16%) of these accounts were institutions. This category consisted of universities and other organizations (e.g., a research center, a public corporation) that offered and tweeted about MOOCs. On average, these institutions contributed 28.4 tweets (SD = 23.3; min = 10; max = 92) to 2.17 hashtags (SD = 1.95; min = 1; max = 10).

  • Course-dedicated accounts. Less than a tenth (18, or 7%) of these users were accounts that were dedicated to particular MOOCs, such as a Humanities101 account for an introductory Humanities course. I used a separate category for these types of accounts because they are qualitatively different than instructor accounts: these accounts focus on a particular course, while individuals with instructor accounts perform many functions, with instruction being just one of them. On average, these course-dedicated accounts contributed 64.5 tweets (SD = 67.9; min = 13; max = 289) to 1.11 hashtags (SD = 0.32; min = 1; max = 1).

  • Platform Provider. The MOOC platform provider (1), ManyMOOCs, was the only account included in this category. I considered the platform provider to be qualitatively different than the institutions offering MOOCs. Overall, the MOOC platform provider contributed to the most hashtags (99) and provided the most tweets (614).

  • Other. Some users (4, or 2%) did not fit into the categories above or include enough information to allow for categorization, and were thus included in this catch-all category.

The box-and-whisker plots in Figs. 2 and 3 summarize the number of tweets posted by user type and the number of hashtags used by user type, respectively. The provider account is not shown because an n of 1 will generate only 1 data point. “Other” accounts are not shown because they represent a catch-all category. Figures 2 and 3 reveal that for all roles, more users posted a few tweets than posted a lot, and more users posted to a few hashtags than posted to a lot.

Fig. 2
figure 2

A box-and-whisker plot depicting the quartiles of posted tweets by user type

Full size image
Fig. 3
figure 3

A box-and-whisker plot depicting the quartiles of hashtags posted by user type

Full size image

RQ 3: what is the content of the tweets tagged with these hashtags and how does it vary by users’ role?

To examine what messages were posted to the course hashtags and how those varied by user role, two researchers coded a random sample of tweets (n = 2251). This sample size was large enough to ensure that results were generalizable to the entire population of tweets posted in 2015 to all hashtags generated by ManyMOOCs with a 95% confidence level and ±2% confidence interval.

First, the researchers used the user categories generated in RQ2 to assign roles to each of the users who were included in this random sample of tweets. Of the 2251 tweets, approximately 35% were generated by learners, 30% by instructors, 16% by institutions, 12% by MOOC-dedicated accounts, 4% by the MOOC provider, and 2% by other accounts.

Next, an iterative process was used to generate codes describing posted tweets. The study by Liu et al. (2016) provided a starting point for coding the tweets used in this study. We chose to use the Liu et al. study as the starting point because the study (a) provided descriptive statistics associated with the different codes identified thus enabling us to make comparisons, and (b) presented a clear explanation of the coding process employed. Using the seven most popular codes generated by Liu et al., the two researchers independently coded 100 tweets. They then discussed the tweets and their associated codes to ensure a common understanding of the meaning of each code and to investigate whether additional codes were necessary to describe the data. At the end of this process, the researchers revised the codes to better capture the data and generated a codebook of 10 codes, which were then used to categorize the full sample of 2251 tweets. The purpose of most of the tweets was to promote, instruct, or share resources. Table 1 presents these codes, their prevalence, and descriptions and examples of each.

Table 1 Tweet categories, percentages, descriptions, and examples
Full size table

Table 2 reveals the types of messages posted by each participant role, ranked in order of their prevalence within each role. The table shows that more than half of the tweets posted by instructors were instructional in nature or contained links to resources, that institutions and MOOC-dedicated accounts performed some instructional functions, and that learners posted very few instructional messages. Reflecting each participant type’s role, Twitter hashtag participation fit traditional educational expectations and functions.

Table 2 Percentage of types of tweets posted by user category
Full size table

One of main activities of non-learner accounts was to post promotional messages. Posting promotional messages was the chief function identified in the tweets of the MOOC provider and institutional accounts with more than 87% and 33% of posts respectively. Such messages constituted approximately 28% of the messages posted by MOOC-dedicated accounts and 16% of those posted by instructors.

In contrast, more than 66% of learners’ posts shared resources, reflections, or feelings. Although learners engaged in some social and educative functions as revealed by their posting of some instructional messages, asking some questions, providing some examples, and introducing themselves, these activities constituted the minority of their overall posts and appear to be peripheral to their participation on course hashtags.

While an investigation of individual user accounts is beyond the focus of this article, it is worth noting that individual participation varied. For instance, our examination of the data revealed that one institution posted the same promotional message on an ongoing basis, one instructor engaged in extensive interactions with course participants, some accounts participated in Twitter chats, and some courses used Twitter as a backchannel where learners could ask questions during a live session.

Discussion

The findings of this study therefore contribute new insights to current knowledge regarding the use of Twitter in MOOCs and MOOC participants’ role differences, and demonstrate the value of large-scale investigations into the use of technology in education in general and MOOCs in particular.

Perhaps most notably, the results show that although social media can afford ample opportunities for interaction, communication, and sharing, most users within the studied MOOC ecosystem used MOOC hashtags in sporadic ways, with the median user posting one tweet to one hashtag. Although more than half of the tweets mentioned other users or referenced web resources, the data provided weaker evidence of interaction between users, suggesting that Twitter use in MOOCs centers on information-sharing rather than the conversational ways that advocates might hope or anticipate. For instance, the proportion of replies in this dataset was lower than that reported in prior research (García-Peñalvo et al. 2015; Liu et al. 2016). The results reported here (a) tend to support the findings of Alario-Hoyos et al. (2014) who noted that participants might want to use the social tools within MOOCs than the social tools outside of MOOCs and (b) begin to challenge common thinking that social media can provide add-on spaces for rich interactions. It is worth noting though, that these findings are situated in a particular context (i.e. the use of Twitter in courses provided by ManyMOOCs). Future research in other settings (e.g., in Facebook or in Twitter hashtags from other course providers) will further elucidate the degree to which the actual use of social media on the ground matches the aspirational rhetoric of social media proponents. An alternative approach for future research may be to study cases in which Twitter (and other social media) are integrated and used in extensive ways in MOOCs, thus identifying factors that lead to successful and fruitful integration of social media in them. Such research might generate further explanations of the results reported here and shed further light on why high social media activity is observed in some MOOCs (e.g., Knox 2014) but not in others.

This study also finds that the existence of course hashtags does not necessarily translate into providing a thriving online space for interaction that might address some of the shortcomings of MOOCs. While providing a course hashtag for all courses might be a promising innovation, these results suggest that the success of this practice may require a more intentional use of the hashtag within courses. To enhance participation and use, platform providers and instructional designers may need to engage in increasing learners’ awareness of the course hashtag and the opportunities it might provide for learning and socializing. The number of users participating in course hashtags pales in comparison to the reported enrolment numbers for MOOCs. Raising awareness among learners may increase their presence on the course hashtags, just as raising awareness amongst instructors, teaching assistants, and staff may increase intentional use of the platform. Other tactics to increase participation in use might include instructors’ referring to Twitter posts in email updates or course discussion posts, or restricting the volume of promotional postings by key stakeholders in favor of postings that elicit participation and interaction. As previous researchers have recommended (Churcher et al. 2014; Liu et al. 2016), greater intentionality is necessary to integrate Twitter within particular courses to achieve particular outcomes, such as in synchronous activities, if it is to be used in the rich ways envisioned by many. On the other hand, it is plausible that the low rates of participation reported here reflect the fact that a majority of learners are not familiar or comfortable with using Twitter in the context of MOOCs. Again, while some research suggests that some users do not view social media spaces as helpful in their learning endeavors, this research is focused on case studies, and the field will benefit from large-scape investigations into the perceptions of MOOC learners regarding the utility of social media for learning.

An important issue to address in future research is the instructional design of MOOCs. Conole (2013) notes that MOOCs can be categorized according to many dimensions, such as for example the degree of communication and collaboration requested of participants. The instructional design of a MOOC is likely to impact the ways that learners use the technological tools that are provided to them, including Twitter. Thus, it might be worthwhile for future research to examine whether social media participation varies according to instructional design. Are there observable differences in how participants use social media according to particular designs? Are the results presented above observable for different types of instructional designs or are they representative of only some types of MOOCs? This line of inquiry raises further questions for future consideration: How does Twitter fit into the ecosystem of tools used within MOOC platforms? Might there have been other rich uses of social technologies that are not visible to this research effort that resulted in a low volume of Twitter activity?

This study also reveals that Twitter seems to serve different purposes for different users: course providers and institutional accounts use it to promote courses; instructors and MOOC-dedicated accounts engage in instructional activities; and learners use it to reflect, share resources, and express personal feelings. Even though these groups share Twitter as a social space and social media have been touted as powerful technologies for enhancing interaction and communication in educational contexts, these results demonstrate that the use of Twitter can be shaped by personal, social, cultural, economic, and political factors. Future research into the influence of these factors may help researchers better understand how they shape the ways that different education stakeholders come to use social media.

Another fruitful direction for future research would be to examine why some MOOC users participate on Twitter more than others and investigate the distinguishing features of very active users. Furthermore, such research could examine whether and in what ways MOOC users’ participation may vary across disciplines and across social media. Some questions of interest with regards to disciplinary differences may be the following: Are faculty teaching MOOCs in the social sciences using social media more or less than faculty teaching MOOCs in natural sciences? In what ways, if any, does participation vary across disciplines? Some questions of interest with regards to differences across social media may be the following: Are some individuals active on some social media platforms, such as Twitter or Facebook, but not on others? If so, what reasons might lead learners to participate on some social media platforms but not others, and how can MOOC providers and instructional designers enable and support such preferences among learners?

A need for further research into the use of social media by MOOCs is suggested by this study’s finding that the proportion of tweets it coded as promotional (26.7%) is higher than that found by other studies (e.g., Liu et al. 2016). Currently, research has provided only limited evidence on which to compare this finding to the types of messages posted on MOOC discussion boards or on other social media. Might there be something unique about Twitter—such as its primary cultural use to promote products, personalities, and causes—that results in such more than a quarter of posted tweets in this context being promotional in nature? Useful insights could also be provided by research into how participating learners perceive the use of such messages as helpful or off-putting and to what extent those messages achieve the senders’ intended outcomes.

Conclusion

This research used a large-scale data set to investigate participation on course-dedicated hashtags. It examined the participation patterns of hashtag participants, the types of users posting to those hashtags, the types of tweets that were posted, and the variation in types of posted tweets across users. While popular narratives suggest that social media provide a space for enhancing learner participation, this study provides little evidence to support these claims in the context of Twitter as an adjunct to MOOCs, finding that an active minority of users contributed the preponderance of messages posted to Twitter hashtags and that learners make up only about 45% of users. Nor do these findings reveal substantive evidence of learners contributing to multiple hashtags, which may suggest that learners did not find Twitter to be a useful space that provided added value or responded to their needs. Ultimately, these results demonstrate the need for greater intentionality in integrating social media into MOOCs.