1 Introduction

Along with developments in information and communication technologies (ICT), e-learning has emerged as a new paradigm in modern education system (Czerniewicz and Brown 2009; McGill et al. 2014; Sun et al. 2008). Fallon and ve Brown (2003) defined e-learning as any type of learning, teaching or educational activity, which is facilitated by online networks based on computer and internet technologies. It can be considered as an extension of the concept of distance learning which emerged in the mid-1980s (Aparicio et al. 2017; Hassanzadeh et al. 2012). In parallel with wide spreading of internet and also electronic devices enabling access to the internet, e-learning has been widely used particularly in higher education all over the world (Cidral et al. 2018; Freeze et al. 2010; Liaw et al. 2007; Zhang and Nunamaker 2003). Hence, there has been a significant transition from traditional classrooms to e-learning systems at universities’ undergraduate and graduate programs (Allen and Seaman 2016; Clayton et al. 2018; McGill and Klobas 2009).

E-learning systems enable learning at anywhere and anytime and provide access to information remotely. Additionally, its key feature of providing flexible and personalized learning to learners makes e-learning a highly preferred learning platform among students (Auld et al. 2010; Bhuasiri et al. 2012; Chiu and Wang 2008; Clayton et al. 2010, 2018; Marshall et al. 2012; Peña-Ayala et al. 2014; Viberg and Grönlund 2013). Scholars have previously developed various theories including Davis (1989)’s the technology acceptance model, the theory of reasoned action and the theory of planned behaviour (Fishbein and Ajzen 1975) in order to understand and explain the antecedent factors of the information systems success. In this vein, DeLone and McLean (1992) conducted a review analysis covering the academic articles published in the period of 1981–1987 and developed the information systems success model, which offers a comprehensive framework for assessing the success of the information systems (DeLone and Mclean 2004; Petter et al. 2012; Seddon and Kiew 1996; Seddon 1997).

This study applies the IS success model to an e-learning system and aims to examine the system from students’ perspectives. The paper is structured as follows. The literature review section provides relevant body of the literature on the IS success model and the components of the proposed model. Moreover, the hypothesized relationships are introduced with their underlying theoretical discussions. The methodology section informs about the application of PLS-SEM and presents the analysis of the measurement and structural models as well as hypothesis testing. The final section presents the results of the study, which are further discussed by considering the limitations of the study in the last section of the paper.

2 Literature review

The IS success model is considered one of the most successful theoretical frameworks to explain and estimate system usage, user satisfaction and system success (Guimaraes et al. 2009). There are six components in the basic IS success model: (1) system quality, (2) information quality, (3) system usage, (4) user satisfaction, (5) individual impact and (6) organizational impact. DeLone and McLean (1992) suggested that these six dimensions of the model are interrelated and hence should be evaluated together rather than as independent items. System quality and information quality affect system usage and user satisfaction both separately and jointly. In addition, the amount of system usage can affect the degree of user satisfaction in both directions. While usage and user satisfaction form the basis of individual effectiveness, it is expected that individual performance will eventually have some organizational impact (Petter et al. 2012). DeLone and McLean (2003) proposed an updated IS success model by evaluating the rapid change in information systems, especially web-based applications. Based on their previous work, they incorporated the “service quality” dimension. Hence, the updated model included six dimensions: (1) information quality, (2) system quality, (3) service quality, (4) usage/intention to use, (5) user satisfaction and (6) net benefits. Information quality is a desired characteristic of system outputs such as administrative reports and web pages such as intelligibility, suitability and usability. The system quality refers to the desired features of a system such as ease of use, flexibility, and comprehensiveness. Service quality is the quality of the technical support the system users receive. System use can be defined as the frequency or the degree of the use of the system. User satisfaction is the degree to which users are satisfied with the output they obtain from the system. Net benefits are the contribution that the information system provides at the individual, group or organizational level (Petter et al. 2008).

The IS success model has been widely used to assess the success of different specific systems such as knowledge management, e-commerce, and ERP systems (DeLone and Mclean 2004; Lin et al. 2006; Wang and Wang 2009). When an e-learning system is considered as an information system, perceptions of users (i.e. students in higher education) about the system and the quality of the information, which both affect learning outputs, gain importance. Learning outcomes are influenced by user satisfaction in an e-learning environment, which is a typical outcome measure for the IS success model (Rossin et al. 2009). Feedback received in an e-learning environment is considered as a measure of the quality of the information provided in a classroom. In addition, the balance of skills and abilities necessary for an e-learning experience is also used as a measure of the system quality. Information required by students varies across different courses. This study aims to investigate how an e-learning information system can facilitate transmission of necessary information to learners/students. As a model extensively used in assessing different systems, the basic assumption of the IS success model is user’s voluntariness. However, this assumption is incompatible in the contexts where e-learning system is compulsory for a higher-education course. In this case, students are required to use the system in order to complete their courses, rather than a voluntary-based usage.

Learning activities in an e-learning environment are carried out through web-based applications. DeLone and McLean (2003) argued that web-based applications are well aligned with the updated IS success model. Therefore, in this study, the examination of the success of an e-learning system was built on the updated IS success model. The success of the system is likely to increase when learners perceive the system as a useful tool in their learning experience. In this context, the next sections introduce the components of the updated IS success model and the hypothesised relationships.

2.1 User satisfaction

User satisfaction is a measure of the successful interaction between an information system and its users. User satisfaction is also defined as the level of learners’ beliefs that the information system meets their needs. If a system meets the needs of the users, the satisfaction from the system is very likely to be high. On the other hand, if a system cannot provide the necessary information to users, satisfaction from the system is expected to be low. Existing empirical research has demonstrated that frequently used systems are associated with user satisfaction (Bharati 2003; Freeze et al. 2010). DeLone and McLean (1992) found that user satisfaction has been widely used in measuring the success of an information system. Since the use of an e-learning system is not voluntary, success of the system must be linked to learning outcomes (Gill 2006). An e-learning system will be perceived as successful if students are satisfied with the system and its contributions to their learning activities. Accordingly, the hypothesis is put forward as follows;

  • H1: User satisfaction positively affect system success.

2.2 System usage

System usage is an important construct in measuring system success (Van der Heijden 2004). System usage construct can also be operationalized as “possibility to use” and “intention to use”. The amount of usage and frequency of usage can be given as examples of this component. Moreover, according to DeLone and McLean (2003), the nature, quality and suitability of system usage are important outputs. In this study, system usage refers to nature and an extension of the use of an e-learning system. System usage is likely to increase when the system is perceived as useful, but is likely to decrease when it is perceived as useless (Freeze et al. 2010). In an e-learning system, system usage is mandatory. However, students may think that system usage is beneficial, or they can perceive that it will not have any impact on their learning experience. In addition, if students see system usage as a contribution to the improvement of their performance in the classroom, then the e-learning system is likely to be perceived to be successful. Accordingly, the hypothesis is put forward as follows;

  • H2: System usage positively affects system success.

2.3 System quality

System quality refers to perception of user about a system. System quality in an e-learning environment is measured by the level of different software applications designed for user needs and hardware provided to users. If a user is not aware of network and internet requirements of an e-learning system, it will not be possible for her/him to use the e-learning system. A high-quality e-learning system has characteristics such as availability, usability, ease of learning and rapid access. In addition, a successful e-learning system should be user-friendly and provide useful feedback to learners (Guimaraes et al. 2009; Halawi et al. 2008). DeLone and McLean (2003) argued that system quality has a positive impact on user satisfaction. Especially in e-learning systems where system use is mandatory, user satisfaction becomes more important and confronts as an obstacle to be overcome in order to consider the system as successful. Accordingly, the hypotheses are put forward as follows;

  • H3: System quality positively affects system usage.

  • H4: System quality positively affects user satisfaction.

2.4 Information quality

Information quality is generally expressed as quality of information generated by a system. Desired characteristics of an information system are accuracy, precision, validity, reliability, suitability and intelligibility. In an e-learning system, features such as the correctness of information, content needs, and the ability to transfer information in a timely manner gain more importance (Swaid and Wigand 2009). In addition, information quality is highly related to e-learning system content. It is important to provide necessary information about the purpose of the course to students before starting the course. Student satisfaction is also related to feedback given to students by the system (Rossin et al. 2009). An e-learning environment that is designed for the needs of learners and the content management system’s usage is likely to positively impact learning outcomes and user satisfaction. Accordingly, the hypotheses are put forward as follows;

  • H5: Information quality positively affects system usage.

  • H6: Information quality positively affects user satisfaction.

The proposed conceptual model and hypothesized relationships are presented in Fig. 1. System quality in the model refers to the perception of a user about the performance of the system. The information quality refers to the system output quality. The system usage refers to the degree of the usage of a system, and user satisfaction is the successful interaction between the system and its users. System success also explains that the system is perceived as useful by the users (Freeze et al. 2010; Mohammadi 2015).

Fig. 1
figure 1

Research model

Full size image

3 Methods

3.1 Study sample and data collection

The constructs in the model are measured through borrowing items from the relevant existing scales in the literature. The measures for system quality, information quality, system usage, user satisfaction and system success are adapted from the study of Freeze et al. (2010). All items are measured in 5-items Likert Scale (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree). The online survey was conducted with 144 students of a state university in Rome, Italy, who use e-learning system. The constructs are measured with the students’ self-reported perceptional evaluations about the e-learning system they use for their studies. The survey was conducted in Italian and available online over 14 months to be filled by the students. The questionnaire was initially sent to 1000 students who were registered in the e-learning system and they were repeatedly reminded for participating to the survey.

The university, which is under focus in this research, provides distance-learning courses at both undergraduate and graduate levels. The university started its distance learning programs in 1996 and has been using Moodle as learning management system. Information about each program (i.e. duration of the course, level of the course, course co-ordinator etc.) can be accessed on the university’s distance learning website. Course contents are published only in Italian. The system provides access to web seminars by using their username and passwords in addition to the courses. Students are required to register and choose an available date in advance for each course’s exam. Although the courses are generally made available to students with a fee, some courses are provided for free.

3.2 Sample characteristics

Table 1 presents the descriptive statistics. Students mostly access to online courses via using their laptops. The table also shows the students’ daily use of internet. Out of 144 respondents, 83.3% of the students are female and 16.7% are male.

Table 1 Descriptive statistics
Full size table

Data availability

Dataset and survey questions are available upon request.

4 Results

PLS path modelling was applied for the data analysis. PLS is particularly suitable for structural measurement models with small sample sizes and explorative research aiming to test and validate models (Hair et al. 2012; Henseler et al. 2009; Ringle et al. 2012). Considering the relatively small sample size (n = 144), PLS path modelling was chosen as the most appropriate for the data analysis. A two-stage analysis approach was followed; started with the measurement model assessment to confirm the validity and reliability, and then run the structural model analysis (Hair et al. 2012). The data was analysed by using SmartPLS 3.0 (Ringle et al. 2014).

4.1 Measurement model results

Internal reliability was assessed by using composite reliability (CR) and Cronbach’s alpha. All CR values are above 0.8 ranging from 0.830 to 0.943, which shows that all constructs demonstrate sufficient reliability (Hair et al. 2017; Nunnally and Bernstein 1994). Cronbach’s alpha values of the constructs, except system usage, are above 0.7 ranging from 0.844 to 0.880 (Fornell and Larcker 1981; Nunnally 1978). The system usage construct as an alpha value of 0.592, which is considered within the acceptable range (Loewenthal 2001). The absolute standardized first-order outer loadings ranged from 0.624 to 0.950; all items are above 0.5 with most items exceeding the value of 0.7 (Chin 1998; Fornell and Larcker 1981). The items SU2 and US1r were omitted as the outer loadings were below 0.5.

The validity of the measurement model was examined by using convergent and discriminant validities. All constructs showed AVE values greater than 0.5, ranging from 0.617 to 0.893, confirming convergent validity (Fornell and Larcker 1981). Discriminant validity was examined with Fornell-Larcker Criterion. The results presented in table confirm discriminant validity as the square roots of AVE of each construct are higher than cross-loadings (Fornell and Larcker 1981; Hair et al. 2010) (Table 2).

Table 2 Discriminant validity; Fornell-Larcker Criterion
Full size table

4.2 Structural model results

The structural model was assessed after confirming the reliability and validity of the measurement model. The predictive power of the model was evaluated with R2 scores. The R2 values of system success, user satisfaction and system usage are 0.617, 0.539, and 0.440, respectively, which are all above a moderate level (Cohen 1988; Ringle et al. 2012). The effect size f-squared is evaluated in order to examine the impact of an independent latent variable on a dependent latent variable (Chin 2010). The effect size of system usage on system success (0.052) is found to be at the small level. The effect size of user satisfaction on system success (0.831) is found to be at the large level.

A bootstrapping technique, which allowed assessing the significance of path coefficients was employed (Henseler et al. 2009). A resampling bootstrapping (5000 resamples) of 144 observations was run. The path coefficient from system quality to system usage is 0.562 (t = 5.393, p < 0.001), which supports H3. The path coefficient from system quality to user satisfaction is 0.512 (t = 5.614, p < 0.001), which supports H4. The path coefficient from information quality to system usage is 0.134 (t = 1.209, p = 0227), which does not support H5. The path coefficient from information quality to user satisfaction is 0.278 (t = 2.960, p < 0.01), which supports H6. The path coefficient from system usage to system success is 0.169 (t = 2.314, p < 0.05), which supports H1. The path coefficient from user satisfaction to system success is 0.678 (t = 13.118, p < 0.001), which supports H2 (Table 3 and Fig. 2).

Table 3 Assessment of the structural model
Full size table
Fig. 2
figure 2

Assessment of the structural model

Full size image

5 Discussion and conclusion

The use of technology in learning environments has increased in parallel with technological developments. Technological devices nowadays provide a flexible learning environment through alleviating time and spaces barriers. Accordingly, studies aiming to facilitate the use and development of e-learning systems have gained importance. This study investigated an e-learning system based on the IS success model (DeLone and McLean 2003) through using student perspectives.

An e-learning system used in a state university in Italy is evaluated through applying the IS success model’s key components; system quality, information quality, system usage, user satisfaction and system success. The results obtained through applying PLS-SEM analysis demonstrated that system quality has positive and significant impact on both system usage and user satisfaction. Regarding system quality, it can be argued that the features of the system such as its continuous accessibility, interactive and user-friendly interface increase system usage and lead to higher user satisfaction. The results also show that information quality has a significant impact only on user satisfaction. This confirms that students believe the system provides the necessary information to them clearly and significantly, which in turn increase their satisfaction with the system. Additionally, both system usage and user satisfaction have positive and significant impact on system success. Whereas the impact of system usage on system success is weak, user satisfaction significantly affects system success. This can confirm that students are satisfied with the system and its contribution to their learning experience and hence considering the system as successful. The high degree of explanatory power of system success (R2 = 0.617) confirms the success of the model in explaining the factors affecting system success. R2 values of user satisfaction (0.539) and system usage (0.440) also confirm importance of system quality and information quality for high levels of user satisfaction and system usage. The findings are in line with the results of the work done by Freeze et al. 2010.

The finding of the present research suggests that the features of the e-learning system affect system success. In addition to system characteristics, investigating readiness of students to e-learning, study discipline and other cognitive factors is important to understand the reasons for preferring e-learning systems by students. Instructor and content developers can also be included in research sample to deeply understand the factors affecting system success from different perspectives. Future studies can include other factors such as service quality and net benefits in the model. Service quality can be considered as technology support provided by the university and accessibility of the instructors in the content of e-learning system. Net benefits can be considered as factors that can enhance students’ learning experiences.

As any other research, this study has certain limitations. The results are derived from a sample of students from a single state university in Italy. This model can be tested in other contexts such as private universities, universities in other countries and also different e-learning systems. It is also important to investigate the role of cultural context in e-learning experiences. Hence, comparative studies would be particularly useful. Moreover, this research applies a cross-sectional approach; future longitudinal studies are also encouraged.

This study relies on subjective measures, as the underlying data was derived by asking the students’ self-reported perceptions on the e-learning tool that is compulsory for their studies. This approach provides important understanding about the success of the focal e-learning tool from the users’ perspectives, as the education systems cannot be measured in isolation. However, the author also calls for future research, which can test the model or e-learning system success with objective measures and data.