Keywords

1 Introduction

In light of the latest technological advances and recent developments in higher education, postsecondary institutions are faced with a number of challenges [1]. At the forefront of these challenges is the utilization of Learning Management Systems (LMS) and the various tools and functionalities provides. Currently, LMS environment is limited to posting material and communicating with students. LMS tools and functionalities should be utilized in assessments and evaluation to enable linking course assessment to expected outcomes [2,3,4]. In addition, and with the increased demand on detailed analysis of achievements of various courses outcomes, LMS can contribute to this since it has tools that track and keep detailed stats of students’ activities and interaction in any course component posted on the online course page. A second challenge facing higher institutions is the increased demand coming from accreditation boards that require detailed analysis of student performance; especially those related to achievement of course outcomes and course objective. These exercises can be rigorous and require tracking various activities and calculating students’ achievements based on their performance on various assessment tools. A third challenge facing higher education institutions is the increase in education cost, which forces institutions to look into ways to reduce education cost without compromising the quality of the education these institutions deliver [4]. In the three listed challenges, technology has been viewed as it may provide the solution to some of the problems associate with these challenges. In addition, researchers have identified the widespread and the exponential growth in the use of smart devices to have a great potential to help educators transform and improve educational methodologies and approaches including course assessment and evaluation [5]. In addition, in recent years the use of Internet in education has grown considerably. However, in many institutions in the region the use of the Internet is limited to providing access to students to course materials and communication with students. While there are many additional tools provided in LMS solutions, a typical course page is usually limited posting course outline, lecture PowerPoint slides, some instructional materials, and communicate assessments results over grade book [6].

In past two years, the University of Sharjah (UOS), United Arab Emirates, has included in its strategic plans expanding the use of LMS tools and functionalities to include online assessment and measuring achievement of course outcomes based on detailed analysis of students’ performance in every assessment tool. This can be easily achievable by linking course objectives to assessment tools for every topic listed in the course outcomes. Technically, such tasks can be easily achieved with the recent web-based and user-friendly LMS systems and solutions that include assessment packages and secure Internet testing protocols. The latter led to increase in employing LMS solutions to conduct summative online assessment, including assignments, quizzes, and tests. Deans and administrative management at the University of Sharjah are enthusiastic about potential for conducting assessments (quizzes, midterms and final exams) using available LMS tools for summative assessment materials. This is mainly due to the various advantages employing online assessments systems bring, including saving time in grading, recording and producing statistical reports for feedback and program evaluation [7].

From our brief search in the literature, it seems that there is no clear consensus among researchers in the field on the impact of online exams using web-based assessment LMS tools on students’ performance [8, 9]. In addition, little information has been found on the students’ perception, acceptance and attitudes towards such online systems. In the UAE and Gulf Region, no studies have been found on this topic, which makes our study the first study that explores the impact of online exams on students’ performance and gauges their attitudes and acceptance (or feelings) towards summative online assessment. This makes the objectives of this initial study to: (1) Conduct a statistical comparison between the performance of the students’ summative (graded) assessment using the online web-based system vs. those assessed using traditional paper-pencil in class exams; (2) Gauge students’ perception, possible advantages, disadvantages and the challenges faced when conducting assessment online via LMS systems. Our goal is to assess the experience of the University of Sharjah with online summative assessment and provide feedback and recommendation to improve the experience and make sure that it serves the purposes it is intended for including improving the education quality through improving courses assessment that is based on expected course outcome. Implementation of such approaches will improve teaching and learning in large-size undergraduate classes through improved student confidence and increased instructional time.

2 University of Sharjah Online Summative Assessment

2.1 General

Before we present the details of our summative online testing, it should be noted that summative testing in an online testing environment that includes various types of course assessments, which were conducted and graded automatically through the LMS system. By definition, summative assessment generally takes place after completing a period of instruction and it requires students to answer questions or solve a set of problems that is based on the covered material to ensure learning outcomes are achieved [10]. The assessment (or the test papers) is graded and counted towards the final grade of the course. In contrast, in formative testing or assessment students are evaluated during the work process and the focus is on improving the process [10]. Online summative assessment process requires high levels of access and security controls that allow students to access and respond to questions through private and carefully generated passwords that are given to students with access limited to LMS testing environment only. In addition, the online summative system must be reliable to ensure accuracy, validity in scores and most importantly free of any technical errors and glitches. Additional security measures are taken by the technical supporting team to ensure that testing is done according to accreditation requirement and course outcomes and objective.

Some of the advantages of the online summative testing, which the University of Sharjah hopes to exploit include [11, 12]:

  • Flexibility in delivering tests to students: Students can write the exams at different times of the day to fit their schedule. This will save the university the effort they endure in scheduling large-size classes in theaters and large exam halls. Less invigilators are needed since students have the option to write the test in computer labs available throughout campus.

  • Efficiency in scoring, recording and reporting grades, which are done automatically via the LMS tools.

  • Since the exams are conducted outside the class time, an additional benefit that can be gained by conducting exams online is the fact additional class time may be gained in traditional on-campus courses. That is, rather than taking a class period for completing the quiz or exam, instructors can use the class period for instructional delivery or other activities.

  • Once the assessment questions are setup the first time, the material can be recycled again several times thereby saving instructors a lot of time and effort.

  • Through technological solutions, cheating can be minimized, especially in MCQ type of exams. LMS testing tools provide randomization functionalities that allow instructors to make multiple versions of the exams so that students have no hopes of seeking assistance from peers sitting next to them. In addition, lock browsers’ functions and applications were installed on the PC’s to prevent students from consulting Google for answers.

While the above advantages were taken to support conducting quizzes and exams online, it should be noted that there are counter arguments that are not as supportive. For example, statistical analysis reported by Hollister et al. and Summers et al. have revealed that conducting exams online had no significant differences in mean exam scores for students [13, 14]. Hollister et al. also found significant variations in scores of unproctored online exams, which were attributed to possible increase in cheating. On students’ satisfaction, Summers et al., reported that students were significantly less satisfied with the course than the traditional classroom students on several dimensions.

2.2 Exam Structure and Setup

As an initial phase, the University of Sharjah introduced online exams to large class sizes, which included university and colleges elective courses. The enrollment of such courses reaches as high as 2000 students. Traditionally, midterm and final examinations for these courses is a logistical nightmare for schedulers, instructors and departments. It involves an army of invigilators and making up several versions of MCQ exams that are photocopied and distributed in a very tedious process to ensure safety and security of the exams. Grading is usually done by hand, which usually requires a considerable amount of time to be finalized, double checked and approved.

Moving to online exams, course instructors were required to submit a bank of questions with various levels (easy, medium and hard) along with model (or correct) answers. The technical IT team uploaded these exams into the Blackboard under a separate course page. Multiple versions were prepared from the test bank with proper randomization parameters that ensures enough versions in specific setting (computer lab). A schedule of the exam halls timing and availability were sent to the students prior to the test date via the Blackboard. Screen shots of the exams for a number of courses are shown in Fig. 1. As mentioned above, students are given the option to sit in any of the announced times that fit their schedule. Invigilation was assigned to IT lab supervisors to ensure the availability of technical support when and if needed. Students were given time limit, which was set by the academic instructors. The number of attempts was left open to students but once they submit, they cannot change their answers. Instructors were asked to conduct mock exams to ensure that students are well acquainted with the system functionalities. Student grades were recorded directly into the students’ course evaluation worksheet.

Fig. 1
figure 1

Two screenshots of typical online exam questions

Full size image

The courses, which exams (Midterm or Final) were conducted online are listed in Table 1. It should be noted that while we have been monitoring the process for the Fall and Spring semesters, the analysis and results reported in this paper were conducted on the Spring 2017 cohort. The first trial (Fall 16/17) was somewhat an exploration of the system, especially from the logistical and technical aspects. The College of Sharia and Islamic Studies takes the credit for taking the initiative. The proposal was submitted to the Deanship of Academic Support Services (DASS) by the Sharia College Council as a solution to deal with the large classes, especially Islamic Culture, which is a compulsory university elective course that must be taken by students in all academic programs. In a typical semester, over 1500 students divided among 40 plus sections usually take the course. The experience proven to be successful providing the much needed help with invigilation, grading, recording, and reporting and course statistics. Building up on the success of the College of Sharia successful experience in the Fall 16/17 semester, more departments submitted requested to DASS to conduct their exams online.

Table 1 The list of exams conducted online using Blackboard on exam tools
Full size table

3 Results and Discussion

3.1 Comparing Students’ Grades: Online Versus Traditional

Before we present the results of the students’ perception and satisfaction survey, we will present a comparison of the students’ grades in one of the courses that was assessed using online summative approach and compare it to students’ grades and class average in previous semesters when assessment was done using traditional paper-based. The purpose of the comparison is to look for possible effects on the overall grade distribution and class averages. An example of such comparison is shown in Figs. 2 and 3.

Fig. 2
figure 2

The distribution of class grades (in Islamic Culture) using paper-based exams (Fall 15/16 and Spring 15/16) versus online exams (Spring 15/16)

Full size image
Fig. 3
figure 3

The distribution of class grades (in and Analytical Biography of the Prophet) using paper-based exams (Fall 15/16 and Spring 15/16) versus online exams (Spring 15/16)

Full size image

As highlighted in the legends in Figs. 2 and 3, Fall 15/16 and Spring 15/16 represent traditionally examined courses, while Fall 16/17 is the semester during which the students were examined using the online approach. Furthermore, the data presented in Figs. 2 and 3 represent the students’ grades in Islamic Culture and Analytical Biography of the Prophet. Both of these courses are offered by the College of Sharia as University Elective Courses.

It is apparent for Figs. 2 and 3 that the grade distribution of the classes is comparable in both approaches, i.e. traditional paper and pencil exams versus online exams. The only observable difference is in the number of A’s, which seems to be lower for courses where the exams were held online. While we could not identify possible reasons for this noticeable difference, we have raised this with the Dean of the College of Sharia and the Department Chairman and they both agreed to monitor this for the next few semesters to make sure that it is a real effect. The passing percentage and the overall class averages are consistent throughout the three semesters. The clear conclusion that can be drawn from Figs. 2 and 3 is that the overall performance of the students did not change as a result of conducting the exams online. This is welcome news considering the many advantages the system provides including logistical, saving time in grading, efficiency in recording and reporting grades.

3.2 Assessing Student Perception of Online Summative Assessment

In this section of the paper, we present the second part of our study, which was conducted to gauge student perceptions of online summative assessment. An online questionnaire sent to students who took the online exams through the Blackboard LMS emailing tool. The questions included in the survey questionnaire were collected from various studies published in the literature [15, 16]. While the total number of students who took the online exams exceeded 5000 students, it should be noted that many of them are taking more than one course, which means that they could be were counted more than once. To avoid that, before performing the analysis, we have run a script to eliminate all duplicate answers.

Among these 5000 students 1493 students responded to the online survey, which is substantial number that should provide significant statistical results. The distribution of the students among the colleges and various courses is summarized in Tables 2 and 3. The selection of the classes was based on the available data for those who took the online exam. In future studies, we plan to focus on specific subjects to limit the control factors. In addition to demographic questions, the questions in the survey focused on asking students about the effectiveness, validity, reliability and security of the online system, including technical issues. They survey also asked students questions about the fairness and the benefits that it can bring to teaching and learning. In addition, questions on the effectiveness of the approach and its impact on their academic performance with the aim of identifying possible risks associated with online assessment from the students prospective. Figure 4 shows sample of the questions, which were provided in both Arabic and English.

Table 2 The distribution of the students who took the online exams over the various colleges
Full size table
Table 3 The distribution of the students who took the online exams over the various courses
Full size table
Fig. 4
figure 4

A screenshot of the questions included in the online students’ satisfaction survey

Full size image

To gauge the students’ perception on the impact of online exams on their academic performance, subjects were asked if “immediate feedback will help me improve my learning experience”. The results showed that 69% of respondents agreed with the statement. In a second question on the same category of questions, the students were asked I they felt “my marks will not be negatively affected by the Online Exams”. Their responses showed that 56% of the students agreed with the statement. Responses were much higher in favor of technical advantages of online exams, especially in questions on accessibility (83% agreed), test reliability (81% in favor) and grading accuracy (84% agreed that grading in online is much more accurate than paper-based exams). Students felt the online exams were easy to navigate (79%) and user-friendly (87%). On the technical assessment questions of the survey, only 23% of the students reported experiencing technical issues during the exam. In addition, we have examined the data for possible gender gaps and there were no consistent visible differences in student responses among genders.

4 Conclusions

In the first part of this paper, we have presented the results of a study conducted at the University of Sharjah, United Arab Emirates, to study the impact of online exams on students’ performance in comparison to paper-based exams. In the second part of the paper, we investigated the students’ perception and attitudes towards taking exams online, with focus on a number of educational and behavioral factors.

The results of the first part of the investigation showed that students’ performance was not affected by taking exams online. The grade distribution and the class passing percentage are consistent for both paper-based and online exams. The only noticeable difference is the slight decrease in the number of students who scored A’s in online exams; an observation that needs to be further examined to ensure that it is real and the possible reasons behind it.

In the second part of the paper, the overall feeling and perception of the students towards online exams were investigated using an online survey questionnaire. While almost half the students enjoyed taking exams online, the majority of the students praised online exams when it came to fairness, security, grading and cheating possibilities. On stress and impact of their academic performance, slightly more than half of students reported positive impact on both dimensions. The latter is in line with the findings of other researches who have reported that students’ satisfaction with online exams is low [13, 14].

The results indicate that there are challenges, including awareness among students, which need to be overcome before the University of Sharjah decides to adopt online summative assessment. Our final conclusion can be summarized in the following statement: even though online summative assessment have proven to provide a number of logistical and academic advantages, the University of Sharjah needs to work on raising the awareness among staff and students to highlight these advantages and additional services. UOS needs to highlight that online assessment in principle should positively impact the performance of students through linking questions to expected course outcomes. Future studies will focus on investigating possible areas where the University should focus on to improve the experience.