Introduction

In recent years, rapid improvements in computers and information technology (IT) have drastically changed the field of education. In particular, starting with the 1960s, society has increasingly shifted from face-to-face or computer assisted instruction (CAI) to online learning (Hsu 2007). CAI have replaced older, gigantic computer systems used in traditional learning, and with increased accessibility to personal computers, teachers now use more multimedia, hypermedia and Internet resources in teaching students.

The phrase “e-learning” was first coined in 1996 by the American Society of Training and Development (ASTD), while proposing the first Internet training course. In 2000, SCORM 1.0 (Sharable Content Object Reference Model version 1.0) was released, and the development of e-learning reached its peak. Then, Stephen Downs proposed “e-learning 2.0” in 2006, which used new technology to develop a shared model that provides greater interaction between users and communities. Furthermore, in recent years, several other innovative learning concepts have been introduced, such as “M-Learning” (mobile learning) and “U-Learning” (ubiquitous learning). As e-learning has expanded, researchers have found that e-learning techniques can compliment and improve the efficiency of traditional learning methods (Chen and Wang 1997; Chen et al. 2004).

Scholars who research e-learning use different methods to explore its evolution. In order to prevent information overload and disorientation to learners, Chen et al. (2008) built a concept map of the domain of e-learning methods by collecting related journal articles and analyzing the field’s development from 1999 to 2004. Shih et al. (2008) applied content analysis to educational articles published in five Social Sciences Citation Index (SSCI) journals from 2001 to 2005 to study e-learning developments and future trends. In both cases, these studies offer in-depth understanding of e-learning development to other researchers.

In some ways however, the studies above are limited. Chen (2005) categorized e-learning into three areas: technology, management and education. These studies however, fall specifically within the category of education; they focus on educational journals and seminar materials. While Chen et al. and Shih et al. provide a useful e-learning structure to help us understand the field, their work is unable to help evaluate the relationship between authors and their research topics.

Since e-learning is multidisciplinary, it is possible to explore the field from the perspective of management information systems (MIS), which involves issues surrounding the roles of both technology and management. Author co-citation analysis (ACA) is a bibliometric technique that provides an understanding of the intellectual structure of disciplines (White and Griffith 1982). It has been applied to understand intellectual structure in many fields, such as information retrieval (Ding et al. 1999), knowledge management (Subramani et al. 2003), semiconductor analysis (Tsay et al. 2003), strategic management (Nerur et al. 2008), and stem cell research (Zhao and Strotmann 2011). However, the ACA method has not yet been used to analyze e-learning development. This study applies ACA to analyze the collected literature on e-learning, with the goal of identifying the main issues of e-learning, and to provide a reference framework for researchers, educators, and learners.

The objective of this paper is to construct an intellectual framework in the field of e-learning. We use the ACA method to analyze articles published in Taiwan between 1996 and 2009 by 70 journals from prior studies of MIS journal ranking and the National Digital Library of Theses and Dissertations (NDLTD). The main research purposes are:

  1. (1)

    To explore the intellectual structure of e-learning from 1996 to 2009, and to provide a better understanding of this structure from the perspective of MIS.

  2. (2)

    To examine the relationships between authors and their research topics, and to discuss similarities among various foci in different studies between international and Taiwanese research.

Literature review

E-learning

E-learning is defined as the lessons of training and learning delivered on a computer via the Internet, intranet or CD-ROM. It can be self-paced or instructor-led, and designed for a variety of applications, from providing company training to distance learning courses (Clark and Mayer 2007).

Important advantages of e-learning are that learners are not limited by time or physical location, and they have full control over the learning pace. Additionally, materials can be both shared and reused. Together, these advantages grant e-learning techniques valuable flexibility. However, while some e-learning environments are well established and controlled, there are still many e-learning websites with too much information that can easily overwhelm and disorient learners (Chen et al. 2008).

In exploring these advantages, scholars have applied different methods to examine e-learning development and its major dimensions. Khan (2001) proposed a framework for e-learning designed to create a meaningful worldwide learning environment. His framework consisted of eight dimensions: institutional concerns, pedagogical, technological, interface design, evaluation, management, resource support, and ethics. Each dimension has several sub-dimensions, and every sub-dimension includes issues focused on a specific aspect of an e-learning environment. Hsu (2007) utilized the bibliometric method to explore research issues on e-learning in Taiwan, and re-categorized the field into seven dimensions based on Khan’s original framework. Hsu then outlined four periods of e-learning development, derived from articles in Taiwanese journals, which she arranged into various dimensions. Table 1 represents the results of Hsu’s study.

Table 1 The periods and dimensions in e-learning
Full size table

Additional examples of scholars interested in e-learning include Shih et al. (2008), who conducted a cognitive content analysis of e-learning from 2001 to 2005. In their study, 444 articles were used as samples and cross-analyzed by publishing year, journal name, research topic, and citation count. The researchers defined seven categories for e-learning, each with several subcategories. These major categories include motivation, information processing, instructional approaches, learning environment, prior knowledge, metacognition, and cognitive psychology. Altogether, they found that most published research on e-learning fit within three areas: learning environment—interactive learning environment; instructional approaches—collaborative learning; and metacognition—perception and awareness. Shih et al. also argued that educators may need to pay more attention to studying teachers’ and learners’ motivations in e-learning environments, while researches should increase their focus on the impact of e-learning on cognition and memory.

Concept mapping has also been used to analyze existing e-learning studies. Chen et al. (2008) used concept mapping to organize core knowledge sources within the field from 1999 to 2004. They designed a query-based interface, where users can select journal and conference articles by data sources, time period, numbers of nodes, and keyword lengths to build conceptual diagrams of these articles and their interrelationships. Their system thus allows learners to see the overall structure of the discipline and identify key articles or themes.

Overall, while the authors listed above offer interesting ways of dissecting and analyzing existing research on e-learning, each of their techniques has significant limitations. In the case of Hsu and Shih et al., their final products are both static—requiring manual updates as writing in the discipline expands—and vulnerable to subjective interpretation. In contrast, while Chen et al. uses a more dynamic way to display e-learning research, his methodology lacks any clear thematic organization. Given these problems in previous work, this study applies ACA, a more objective research technique designed to collect both dynamic data and cluster authors into thematic groups.

Author co-citation analysis

Co-citation analysis is one of the most common tools for investigating the intellectual structure of an academic discipline (Acedo and Casillas 2005; Ma et al. 2009). By treating bibliographic elements as conceptual units, the technique helps researchers analyze discipline structure and reduce personal bias within their results. Co-citation analysis is based on tracking the number of times that two authors or documents are cited together. ACA assumes that the more frequently two authors are cited together, the closer the relationship is between them (White and Griffith 1981). Based on author co-citation frequencies, ACA makes a prospective methodology for understanding the evolution of an academic discipline (White and McCain 1998). In general, ACA contains the following steps (Nerur et al. 2008): (1) identify authors highly cited by research articles; (2) retrieve co-citation counts for each pair of authors; (3) compile a matrix of raw co-citations; (4) perform clustering through various analytical methods (e.g. multidimensional scaling, MDS); (5) interpret the results.

While ACA has been applied to many fields (Acedo and Casillas 2005; Ding et al. 1999; Ma et al. 2009; Nerur et al. 2008; Subramani et al. 2003; Tsay et al. 2003; Zhao and Strotmann 2011), it has not yet been used to analyze e-learning development. Since a purpose of this study is to map the intellectual structure of e-learning, we adopted ACA and the steps from Nerur et al. (2008). Furthermore, we also use both MDS and hierarchical agglomerative clustering (HAC) methods to provide graphical representations of author proximities. Table 2 compares our approach with previous studies.

Table 2 Previous studies of ACA
Full size table

Methodology

Rooted within the perspective of MIS, this study develops an intellectual structure for e-learning. The following section outlines our methodology (Fig. 1).

Fig. 1
figure 1

The procedure for constructing intellectual structure of e-learning

Full size image
  1. (1)

    In the first stage, in contrast with the methods mentioned in Chen et al. (2008) and Shih et al. (2008), which used educational samples, we sample from information management papers, including international MIS journals and Taiwan’s NDLTD.

  2. (2)

    Authors’ data processing stage: here, we first extract author data from article reference pages, and then calculate the number of citations for a single author. Only the first author of the work is extracted when calculating citation counts. In some cases the same author might be cited differently, for example “C.-S. Ong,” “Ong, C.” and “Ong, C.S.” In this situation, they must be treated as the same to ensure accurate count citations. Additionally, even though one author may be cited many times in one article because of his different works, he is credited with only one citation. In other words, we are only concerned about whether or not the author is cited in the article at all; not how frequently he is cited within the single article. After collecting these citation counts for authors, a threshold is selected based on criteria from Ding et al. (1999) and Subramani et al. (2003) to determine the most highly cited authors. These authors then represent the sample used to calculate author pair co-citation counts at the next stage.

  3. (3)

    Authors with the most citations are used as input units to calculate author pair co-citation counts and form a square symmetric matrix. After this matrix is constructed, MDS and HAC are applied to measure the distance between authors. An MDS map and a dendrogram are both used to visually represent this intellectual structure.

  4. (4)

    According to the MDS map and dendrogram, the authors can be grouped into clusters. Some unique properties are also analyzed, such as the common research topics of the authors, major topics of e-learning in the MIS field, and differences between the chosen topics of domestic and international researchers.

Implementation

Sample sources

International journal articles

The MIS journals are reclassified into six categories covering 70 journals, by integrating the categories of journals discussed in Nerur et al. (2005), Rainer and Miller (2005), and Lim et al. (2009). Table 3 shows our categories relative to those from other researchers. We collect articles by searching for the phrase “e-learning” in the title, abstract, and keyword sections of papers published between 1996 and 2009. Altogether, we selected 127 articles from 27 journals for our sample. A list of these articles and journals can be found with this paper’s appendix.

Table 3 Categories in contrast with other researchers
Full size table

Taiwan’s NDLTD

The NDLTD is a searchable online collection of theses and dissertations (Fig. 2), from which users can download full text copies. Here, we use “and” logic with “e-learning” in titles, and “資訊” (information) in the department heading to find writings related to e-learning in the MIS field. Altogether, we found 379 works through NDLTD.

Fig. 2
figure 2

Query website of NDLTD (http://ndltd.ncl.edu.tw)

Full size image

Author data processing

Unify author names

Before data processing author names, we can extract some necessary information from the given samples, such as the sample numbers, the cited authors who were listed as references, and the titles of the cited works. Within the MIS journals, 4,560 references and 3,118 authors were found in 127 articles. In the 379 articles from NDLTD, 7,531 references and 3,356 authors were found. Only the first authors in the citations are included, while other co-authors are neglected. Due to the different formats of the authors’ names, it is necessary to manually inspect these names and unify duplicates. Altogether, 113 author names are unified through manual inspection. Also, Google Scholar is used to help confirm whether similar names refer to the same author.

Program implementation

After unifying the authors’ names, this paper uses a C# computer program to help count author co-citations via the following procedures. Figure 3 displays our interface, connected to an Access database along with an example. The first button selects a raw data file, while the second executes the program, calculating co-citations and producing them within the Access tables.

Fig. 3
figure 3

The interface of implementation

Full size image

Single author citation counts are calculated using the third button within the program, shown in Fig. 4. As noted above, if the same author is cited multiple times within an article, the author is still only rewarded one counted citation. In this study, 127 journal articles were used as inputs, and thus the maximum possible citation count for a single author is 127.

Fig. 4
figure 4

Single author count citations (partial outcome)

Full size image

In order to identify important authors within our intellectual structure, the fourth button in our software program manages the citation threshold used to signify authors with high citation counts. Following standards set by previous studies by Ding et al. (1999) and Subramani et al. (2003), there are 40 authors who match the threshold (greater than or equal to five times) among the citations of international journal articles and 41 authors among the citations of NDLTD articles (greater than or equal to 15 times).

The last feature of our program is to execute author pair co-citation counts. Just as with the rules for single authors, if two authors were cited many times in an article, their co-citation counts do not increase. Figure 5 gives an example. Although Collis and Dick are cited many times in an article for their different works, each is given a co-citation count of one. Figure 6 displays the final results of this process.

Fig. 5
figure 5

An example of author pair co-citation execution

Full size image
Fig. 6
figure 6

Author co-citation counts

Full size image

Forming the intellectual structure

After the author co-citation count table is completed, it is converted into a symmetric matrix. MDS maps and dendrograms are generated using the MDS and HAC algorithm. MDS is a set of techniques used to perform graphical representations from matrices. The MDS maps in this study follow two dimensions. Authors are placed on the maps according to their proximities in the author co-citation matrix, and those with higher similarities are placed closer together. The HAC algorithm is a bottom-up approach to merging data points. It regards each data point as an independent set at first, and then calculates the similarities between them. The closest two points are merged into clusters continuously until the merging process achieves the conditions set by the user. In this study, each author represents one data point. A dendrogram helps detail the relationships between clusters. Consistent with prior research (e.g. Subramani et al. 2003); the Ward’s method is adopted to calculate the distance between authors.

Clustering results for citations of international journal articles

A dendrogram for the results of the 40 × 40 author co-citation matrix shown in Fig. 7 represents each point’s distance-cluster combination. The 40 authors are divided into three clusters: psychological research for using IT in learning, the usage of IT in learning activities, and adaptive web-based learning. The shorter the linked distances between authors, the stronger their conceptual relationship. In addition, some categories and sub-categories are classified so that each cluster only presents common research topics to the authors. Table 4 displays these categories and authors.

Fig. 7
figure 7

Dendrogram of authors in citations of international journal articles

Full size image
Table 4 Clusters, categories, and their corresponding authors from international journal articles
Full size table

In the cluster of psychological research using IT in learning, there are 24 authors classified into three categories: user behavior and acceptance (eleven authors), social cognition and self-efficacy (six authors), and technology acceptance model (TAM) and satisfaction (seven authors). Within these clusters, it is clear that the authors’ works are mainly focused on users (especially learners) in the e-learning system. They are concerned about learners’ behavior, acceptance, and satisfaction of learning with IT. Moreover, some authors applied social cognition theory, proposed by Bandura (1986), to examine what and how many factors affect learners’ behavior in e-learning environments.

Within the cluster of the usage of IT in learning activities, nine authors are classified into two categories: design of e-learning and cooperative learning. In several cases, authors conducted their researches through a constructivist view, assessed the e-learning framework they proposed, investigated cooperative learning in e-learning environments, and discussed the impact of learning through IT.

The third cluster, adaptive web-based learning, has seven authors focused on developing adaptive learning on the web. These authors discussed the suitability of various teaching techniques in e-learning environments. In particular, they examined the idea of semantic webs and used item response theory (IRT) to personalize learning systems.

Figure 8 shows the completed MDS map, tracking authors according to their “research viewpoint” and “research method.” Here, it is noted that most authors focused their work through the learners’ viewpoint; their studies focused on learners’ mentality and behavior while using IT in a learning environment. Furthermore, the studies seem roughly divided evenly between theoretical and practical applications.

Fig. 8
figure 8

MDS map of authors in citations of international journal articles

Full size image

Clustering results for domestic citations of NDLTD

Figure 9 represents the 41 authors’ distance cluster combinations. The overall cluster compositions are fairly consistent with the clusters obtained through MDS as given in Fig. 10. Table 5 provides categories and sub-categories of common research topics for these authors.

Fig. 9
figure 9

Dendrogram of authors in the domestic citation of NDLTD

Full size image
Fig. 10
figure 10

MDS map of authors in the domestic citation of NDLTD

Full size image
Table 5 Clusters, categories, and their corresponding authors from domestic citations of NDLTD
Full size table

There are 25 authors within the cluster of implementation of e-learning. They are classified into three categories: teaching theory and online teaching (20 authors), teaching situation building (four authors), and CAI design and establishment (one author). Although the C1 (teaching theory and online teaching) category appears to come from the educators’ viewpoint, some authors conducted research from the learners’ point of view, such as 洪明洲 (Hong, Ming Zhou) and five other authors. Their works mainly focused on learning behavior and effects.

e-Learning, ten authors are classified into two categories: present and prospects, and also introduction, standard and application. These authors primarily worked on the synthesis of e-learning. For example, common trends include research on definitions, standards, future trends, and so on.

Figure 10 shows the end results of the 41 × 41 author co-citation matrix. The 41 authors are divided into three clusters: application in business, implementation of e-learning, and conceptual study of e-learning. Each author is examined along two key dimensions, “research target” (x axis) and “research method” (y axis).

The clearest difference between Figs. 8 and 10 is the x axis dimension. Some of Taiwan’s authors placed their research targets on enterprises instead of school members. They voted to apply e-learning experiences into business training, examining how to promote business innovation, and observed the effects of business with e-learning environments.

Comparisons of research topics

Table 6 provides a comparison between this study and prior research. Altogether, seven dimensions are compared. In prior research on e-learning, Shih et al. and Chen et al. both used international journal articles as samples, while Hsu relied on domestic research. In this study, however, both international and Taiwanese sources are collected.

Table 6 Comparisons of studies
Full size table

Previous studies approach their analysis of the discipline entirely through the viewpoint of education or technology. In contrast, our work focuses on technology and management and examines our results from an MIS perspective. Additionally, both Shih’s and Hsu’s results are static and subjective due to their chosen research methods. Although Chen et al. picked a more dynamic way to present their intellectual breakdown of e-learning research; their methodology lacked the ability to thematically cluster various authors. ACA however, combines the advantages of both sets of authors; it offers an objective approach and a dynamic set of data, while also showcasing the ability to group authors by theme.

International versus domestic authors

Comparing the results shown in Figs. 8 and 10, it is clear that Taiwanese authors place more emphasis on practical research (26 authors in Fig. 10) than their international counterparts. Furthermore, Taiwanese authors examined the e-learning habits of not only schools, but also enterprises. While international authors were more evenly divided between both theoretical and practical issues, their research focused primarily on schools, and the viewpoints of educators and learners.

In Tables 4 and 5, international authors display particular devotion to e-learning and psychological research. Almost all of the international authors pay more attention to user response and acceptance. They adopt theoretical bases for models involving social cognition, self-efficacy, and technology acceptance. Several studies focus on analyzing users’ behavior towards the addition of IT into learning. Some Taiwanese authors, such as 陳年興 (Chen, Nian Shing), 陳育民 (Chen, Yu Min), and 何榮桂 (He, Rong Gui) also compare users’ behavior and adaptive learning between traditional and web-based learning environments. However, most of the Taiwan’s authors focused on the teaching/learning environment from the theoretical perspective of teaching, rather than user reactions. In Taiwan, the implementation of e-learning is the main research focus. With the exception of conceptual studies, several authors took teaching theory as the foundation of their practice and system designs.

Some Taiwanese authors are also devoted to business applications, while international authors put less focus on this area. 黃貝玲 (Huang, Bei Ling) and 廖肇弘 (Liao, Zhao Hong) explore online training to accelerate business innovation, and 呂斌南 (Lu, Bin Nan) and three other authors investigate business models and the impacts of applying e-learning for enterprises.

The similarities and differences between MIS and the educational perspective

We take Shih’s results to the comparative perspectives in MIS and educational viewpoint. From the MIS perspective, the similarities include:

  1. (1)

    The MIS authors voiced concern about user (especially learner) behavior, which corresponds to the category of “motivation: behavioral change” in Shih et al.’s (2008) results. For instance, researchers Ong, C.S. and Chau, P.Y.K. who appears in Table 4 examined user behavior and acceptance in e-learning environments through differences in gender, career, age, and so on.

  2. (2)

    The category of “social cognition and self-efficacy” corresponds to “metacognition: perception and awareness” in Shih’s results. Bandura, A. and five other authors devote time to this area.

  3. (3)

    The category of “cooperative learning” corresponds to “instructional approaches: cooperative learning” in Shih’s results. Slavin, R.E., Johnson, D.H., and Goodyear, P. investigated cooperative learning environments for learners.

  4. (4)

    The MIS authors focus on teaching situation building, which corresponds to the category of “instructional approaches: situated learning” in Shih’s results. 余民寧 (Yu, Min Ning) and three other authors also examined this topic.

  5. (5)

    Both of the two perspectives examine individual learning history to provide adequate content for learners. The topic “adaptive web-based learning” among international authors corresponds to the category of “information processing: individual difference” in Shih’s results. However, although the purposes are identical, the implementation and methods between MIS authors and Shih et al. are different.

  6. (6)

    Although both of the two perspectives examine individual behavior and cognitive psychology, TAM, user satisfaction, and acceptance are stressed in the research topics of MIS authors.

Differences between the two perspectives include:

  1. (1)

    From an educational perspective, some studies discuss technical capability as prior knowledge in e-learning environments because most educational authors do not devote time to technical issues. In contrast, technical capability is not considered as prior knowledge from the MIS perspective, but rather as a known competency. Most MIS authors explore IT applications in e-learning.

  2. (2)

    Concepts of e-learning such as introduction, standard, and prospects are proposed from the MIS viewpoint. It reveals that researchers are intent on understanding the bases of educational characteristics and how IT impacts educational development. In other words, MIS authors are interested in the connection between “e” and “learning.”

  3. (3)

    The research topics of this study emphasize the “instruction” viewpoint, which covers several issues such as the design of e-learning, teaching techniques and the establishment of teaching situations. However, Shih’s results pay more attention to the viewpoint of “learning,” such as with interactive learning, learning communities, and cognitive psychology characteristics.

  4. (4)

    Lastly, some MIS authors are devoted to business applications, which Shih et al. does not cover. Therefore it can be concluded that MIS authors focus on applications of e-learning. For example, they voice concern about IT applications in e-learning environments for staff training and what will be coming for business innovation.

Conclusion

This study applies ACA method to construct an intellectual structure of e-learning. International journals and Taiwan’s NDLTD are two sample sources from which articles were collected. Through the MDS and HAC algorithms, outcomes are presented in two forms: a MDS map and a dendrogram. We analyze these results to determine what research topics are most mentioned within the discipline and who the important authors from 1996 to 2009 are.

Additional results show that research topics from MIS and educational perspectives include individual behavior and cognitive psychology, but TAM and users’ satisfaction and acceptance are emphasized for MIS researchers.

Within MIS, IT is definitely the main topic of research for both international and Taiwanese authors. The development of technology, standards, and the design of learning environments are all emphasized categories. Furthermore, our results suggest that Taiwanese authors focus more on IT rather than education when they examine themes in e-learning.

E-learning for educators and learners to better understand what issues are under scrutiny and which authors are devoted to these topics. It is hoped that this new structure will bring us new applications for tracking e-learning development. For future research, we advocate additional theoretical and practical discussion on e-learning and its structural breakdown.