Online Assessment from a Broader Perspective with Practical Applications

Abstract

Assessment and evaluation can be one the most challenging aspects of online learning processes. With the current global pandemic in particular, the wave of digitization has become a more prominent issue for all level educational institutions. Online assessment is part of the instructional design process and cannot be considered independently of this process. The type of content, learning objectives, and expected outcomes play an important role in determining these assessment methods. In this context, the main aim of this chapter is to explain online assessment and evaluation approaches and to present sample applications for different content areas and which tools and techniques can be better for instructors during the designing of the assessment process. Within this aim, assessment tools and strategies have been presented for the fields of education science, medical education, and legal education, with specific usage cases.

1 Introduction

Assessment is an important component of teaching and learning for all levels of education. For online learning, assessment is also one of the most challenging parts of the learning process. The recent global Coronavirus pandemic has hastened a wave of digitization for all levels of educational institutions. While even in traditional face-to-face learning environments assessment types and techniques play a critical role, it is more challenging in online learning environments. From a general perspective, the main goal of assessment is the process of gathering and interpreting data by instructors for grading and for tracking learning (Arnold, 2014; Boud, 2000). The assessment process comprises several dimensions that affect the entire process, including finding the most efficient method, security, and reliability.

Online assessment is part of the instructional design process and cannot be considered independently from this process. The type of content, learning objectives, and expected outcomes play an important role in determining these assessment methods. The extended use of information and communication technologies and learning management systems (LMS) have necessitated integrated tools for assessment activities. It has been observed that traditional assessment methods such as multiple-choice tests, true/false questions, or drag-drop questions are preferred in systems with many assessment tools (Lourdes et al., 2017; Stödberg, 2012). The assessment process consists not only of grading students, but also the evaluation of the entire process, including the instructor and the program itself. Therefore, there are no one-size-fits-all solutions in an effective online assessment process. For an effective assessment process, instructors should plan to combine both formative and summative assessment approaches. Related to these approaches are several tools, each of which has advantages and disadvantages. Combining various assessment tools can compensate for their individual disadvantages.

In this chapter, assessment strategies and sample cases that can be effective for different fields are presented. The presented cases serve as examples of the methods that can be included in an online course.

2 Theoretical Approaches

Assessment and instructional design cannot be considered separately from each other. Instructional design models such as ADDIE, ASSURE, Dick & Carey Model, and the Kemp Design Model are widely used in the design of e-learning processes. Although named differently, each model contains assessment stages for both the process as a whole and for learning outcomes (Edmonds, Branch & Mukherjee, 1994). Therefore, independently determining the techniques to be used in the assessment process from the content and teaching method leads to problems in the implementation process.

There are two main approaches for online assessment: formative and summative. Formative assessment can be defined as “assessment for learning”, which evaluates student learning and allows for the next step to be planned. Summative assessment can be defined as “assessment of learning”, and systematically evaluates student achievement, mainly at the end of the semester (Harlen & James 1997). Despite appearing different from each other, the two assessment approaches share a relationship, and their combination can provide a sound and reliable assessment process for online learning environments (Fig. 1).

Fig. 1

Formative and summative assessment approaches

The assessment approach widely used in traditional learning environments can be defined as a summative approach to assessment. In general, it is based on obtaining a grade, usually consisting of the average scores of the final exam and other exams related to the course. Formative assessment on the other hand is a feedback-based process that aims to improve learning within the process rather than at the end (Guerrero-Roldán & Noguera, 2018). Both of these assessment approaches are also widely used for online learning environments. Although formative assessment seems to be one of the most effective approaches in the assessment process, situations do arise in which little to no improvement is seen in students, despite regular and clear feedback. It has been reported that these poor outcomes are due to deficiencies in the instructional system associated with the formative assessment (Sadler, 1989).

Online assessment is not an easy process and challenges arise when an effective online assessment process is desired. The two main challenges are instructor workload (Dunn et al., 2003; Sheridan, 2006) and cheating (Alruwais et al., 2018; Fask et al., 2014). When comparing the processes of online and traditional learning environments, online learning requires more effort than traditional learning environments. In parallel, a well-designed assessment process, regardless of online or face-to-face learning environments, is expected to set clear expectations with a reasonable workload and provide opportunities for students to learn, rehearse, practise, and receive feedback on their own. For this reason, it is important that the activities used in the assessment process match the expected learning outcomes (Ragupathi, 2020). While cheating is one important element that should be controlled regardless of whether learning is occurring online or face-to-face, it causes more concern in online learning environments due to the lack of control. Turning this disadvantage into an advantage can be done through an effective assessment design. Research has shown that open-book or open-web-exam environments in particular, with their open resource access, support deep learning (Myyry & Joutsenvirta, 2015).

To evaluate the process in online assessment, presented content should be diversified and the assessment process should be structured using different activities. Students who interact more with the learning activities and tasks in the system will be more advanced in terms of learning and proficiency. Therefore, it will not be effective to conduct formative assessment in a course design when students are not actively participating in the process and are inhabiting the role of the passive listeners (Spector & Park, 2017). For this reason, instructional design and assessment design are two components that interact and are directly interconnected. In this chapter, the authors will provide a holistic implication set for three different fields utilizing real-life cases. The presented frameworks will be applicable in other fields as well (Fig. 2).

Fig. 2

The instructional design and assessment design relation

Online Assessment Design in a Learning Management System

Learning management systems (LMS) contain a large number of content creation and assessment tools. Open-source LMSs, which are open to public development, allow for access to many plug-ins free of charge on the web. Systems such as Moodle, Canvas, ELMS Learning Network, Open Edx and Forma LMS are widely used by educational institutions. Although each LMS has advantages and disadvantages, Moodle is one of the most widely used open-source LMS. The most important aspect that makes Moodle stand out from other LMSs is that, due to its widespread use, many developers offer its plug-ins and share open-source solutions to problems encountered. Many commonly used LMSs share similar tools in terms of content creation and assessment activities. The tools used in the context of assessment activities and their areas of use are given in Table 1.

Table 1 Assessment tools, usage areas and functions

In general, the main assessment tools included in LMSs can be categorized in this way. All these tools can be used for courses from any field according to the course design and the instructors’ planned objectives. However, using all available tools does not lead to an effective and efficient evaluation process, while selecting and using some of the most useful tools in supporting each other will provide more effective results in assessment design.

When it comes to conducting both theoretical and practical courses through online learning, the first option that comes to mind is to design courses in which theoretical knowledge is transferred or reinforced or where theoretical knowledge is reflected in practice. Therefore, from a general perspective, three different course designs can be presented, focused on content, discussion, or collaboration. These design options are presented in Table 2. Online assessment activities in the context of these course designs offer various opportunities to measure the acquisition of theoretical knowledge, the analysis and synthesis of theoretical knowledge, and the reflections of the achievements after application.

Table 2 Course design and related tools

Table 2 summarizes the online assessment tools that can be functional for three different course designs focused on content, discussion, or collaboration. In this context, first consider a case in which all or some of the units of a course are designed with a content-oriented approach. In this design, LMS tools (for example, virtual classroom tools like Zoom, Collaborate, Teams etc.) are used to enable students to interact with the teacher simultaneously. Asynchronous study resources such as videos and presentations/documents are shared on the course page for students to gain knowledge.

In a discussion-oriented design, LMS tools can be used to enable instructor-student interaction, both synchronously (e.g.: virtual sessions) and asynchronously (e.g.: discussion board). Students can discuss a specific topic with both their teachers and peers through various web conferencing tools. On the other hand, it can be ensured that a group or all students can discuss a topic together by using asynchronous tools such as forums. Thus, it is possible to reinforce theoretical knowledge.

In a collaboration-oriented design, students can plan a project study in virtual sessions using various web conferencing tools. Students may be asked to make a video of a product or project process they have designed to observe whether they reflect the theoretical knowledge they have acquired. These videos can be evaluated by peers both within the group and in other groups.

In the next section, course designs for different educational fields and assessment activities related to these designs are presented.

3 Suggested Designs for Specific Fields

The global health pandemic has created a learning emergency necessitating the alteration of learning and assessment strategies (Adedoyin & Soykan, 2020). This emergency has brought about meaningful changes in the assessment approaches of higher education faculty members (Rapanta et al., 2020). For example, the use of certain assessment strategies, including essay exams, oral exams, project work, practical assessments, and portfolios, has decreased (Pandya et al., 2021). In addition, Pandya et al., (2021) determined that while there was a significant difference in the teaching methods and readiness of the faculty members before and during the pandemic period, there was no significant difference in the course content and technological support. This finding shows that faculty members are trying to adapt to the emergency distance education process by changing the learning and assessment methods by increasing their technical skills required for online education without making any changes to the content.

However, in the emergency remote teaching process, it seems difficult to expect digital transformation alongside this pedagogical transformation (Flores & Gago, 2020; Iivari et al., 2020). In this context, supporting teachers with in-service training is the first solution that comes to mind. For example, a report prepared by the World Bank stated that in-service training should be supported both educationally and technologically (Beteille et al., 2020). However, we do not yet know the implications of teachers’ experience for the effectiveness of online assessment design at the end of a process where solid theoretical knowledge can be obtained, and this knowledge is supported by practice. For example, Flores and Gago (2020) stated that ideally presented scenarios do not correspond to real application contexts.

However, solutions to be offered for online assessment should be designed taking into account a controlled workload for instructors and students.

3.1 Case 1: Online Assessment Methods and Techniques that Can Be Used in the Field of Education

Education faculties are institutions in which learning activities are examined in-depth, both theoretically and practically. In addition, the field of educational technology focuses on the design and development of appropriate technology to achieve learning goals. Therefore, with the dynamism of the field of educational technology, it can be expected that education faculties will be better equipped to meet emergency education needs. In this context, it seems possible that online assessment designs proposed for education faculties would be more diverse than those of other fields. This chapter, however, focuses on content-oriented course design in its examples of online assessment processes.

In content-oriented course design, the resource viewing reports can be in the online assessment process, if the number of students enrolled the course is high, multiple choice question tests can be structured using assessment tools such as peer assessment or assignments with rubrics and essays. Since individual student work is more important in content-oriented design, information can be obtained on how long participants stay on the course page (Fig. 3).

Fig. 3

Online time spent in a moodle course (Moodle plugins directory: Attendance Register, 2017)

By examining participation reports, it is possible to identify in advance learners who are likely to have limited interaction with the content, and feedback can be provided to these learners through various tools in the Moodle LMS. For example, reports can be obtained on which resource a student has viewed and when and information messages can be sent through the system to learners who do not interact with the content of the relevant week (Fig. 4). 172.

Fig. 4

Participation report in moodle LMS (Participation report—MoodleDocs, 2019)

If students want to be assessed through homework, Moodle LMS provides functional tools for teachers to score homework. The first of these tools is rubrics, which allow teachers to make categorical scoring of assignments by considering predetermined criteria. A fairer assessment of an assignment or project may be possible using rubrics within the Moodle LMS (Fig. 5).

Fig. 5

Rubrics example for assignment/project in moodle course (Rubrics—MoodleDocs, 2019; Rubrics and Grading Guides, 2021; Rubrics and Grading Guides, 2021)

With activity completion reports in a discussion-oriented lesson or unit design, it is possible to assess whether students fulfil the minimum tasks required from them in the forum (activity completion reports) or to assess their performance based on their activities in the discussion boards (rubrics enriched with learning analytics). When criteria are created on variables such as the minimum number of discussion views, the number of views shared in the discussion forum, or the average score given to the views shared in the discussion, activity completion reports can be used to check whether the student fulfils the basic requirements (Fig. 6).

Fig. 6

Activity completion in moodle (Using Activity completion—MoodleDocs, 2021)

The use of a component such as the discussion board also provides tools to facilitate the peer review processes. For example, it can be ensured that other students or teachers give points to each discussion comment, and the average of the given points is shown below the comment. Therefore, peer evaluation is automatically employed in this process (Fig. 7).

Fig. 7

Peer assessment in a discussion board (Forum activity—MoodleDocs, 2021)

Peer assessment processes can be carried out more easily in a collaborative course design. Using tools within the LMS, students can be grouped randomly or according to a certain systematic criterion (such as the number of students in the group), making it easier for a group of students to work together. Thanks to these tools, each group can have a separate discussion activity or separate task (Fig. 8).

Fig. 8

Restrict access to a resource or an activity by the group (Restrict access settings—MoodleDocs, 2021)

3.2 Case 2: Online Assessment Methods and Techniques that Can Be Used in the Field of Medical Education

Medical education, along with other training areas where practice and clinical skills are at the forefront, is one area facing difficulties in the remote online learning process. The need for hands-on practice is one of the biggest limitations to online education in the medical field, where practical experience with the human body is an integral part of the learning process. This greatly affects the needs in the design of the course. For this reason, these handicaps can be avoided by choosing a blended model instead of completely online education in areas in which clinical skills are at the forefront. Since medical education is both a theoretical and practical course, the content that can be presented theoretically can be transferred online. In addition, with the video-based structuring of the departments in which clinical skills are involved, students will be provided with the opportunity to repeat it many times and when they come together in a face-to-face environment, students will be provided with a preliminary knowledge of these skills. This will enable the transfer of the skill to be explained in a shorter time and the time during the lesson to be allocated to the questions of the students and the development of their practical skills. Of course, this process is reflected in the evaluation dimension in the same way. Structured Clinical Examinations (OSCEs), which are used to assess practice-based skills, or assessment processes conducted with volunteer patients are the main assessment elements that are interrupted in the fully online learning process (Fuller et al., 2020).

For this reason, a video-based course design in the field of medical education supports the achievement of learning goals. Asynchronous videos, lecture notes or handouts, face-to-face sessions, and external resources can be included in a lesson to be designed in this context.

Asynchronous videos can be used to increase the readiness level of students, especially before the lesson. In this context, important terms and topics specific to the related subject should be explained and theoretical information presented in these videos. If necessary, clinical skills should be explained with a shot taken during practice in a second video. Points to be considered in asynchronous videos are limiting video length to 5-6 min and the instructor’s adopting of an energetic and lively narration style during the lecture (Costley et al., 2017; Ilgaz, 2019). Lecture notes or handouts also serve to enrich the course by providing a guide for topics that include theoretical knowledge or practical skills. These articles, best practice videos, or important news can be presented as external resources.

In the assessment process, in accordance with the topic presented each week, peer assessment, homework evaluation, rubric evaluation, question banks enriched with equivalent questions, essays based on clinical case interpretation, tests, usage of rubrics enriched with learning analytics, Q&A sessions, video reflection blogs, video recordings, and group posters events can be given.

In such blended learning environment designs, it is important that the students do the pre-work and come to the lesson ready. For this reason, it may be preferable to include the students’ viewing of videos, lecture notes, or external resources in the system in the course grading, as this will affect participation. Data usage logs can be utilized to obtain activity completion reports (Fig. 9).

Fig. 9

Activity completion report (Activity completion report—MoodleDocs, 2021)

Clinical practice videos uploaded by the students with the rubric to be prepared by the instructor can also be evaluated by the students in a peer assessment activity. For this purpose, the workshop or the peer work add-in can be used. With the workshop add-in, students can assess other uploaded videos according to the assessment criteria prepared by the instructor (Fig. 10). Optionally, it is possible to assess their own videos. Another effective aspect of this assessment method is that students should be able to self-assess according to the given criteria.

Fig. 10

Workshop activity (Workshop activity—MoodleDocs, 2021)

In a course design implemented as described, it can be observed whether each group has completed the assigned tasks and how much progress they have made thanks to the activity completion reports. In this way, separate feedback can be provided for each group. Reflecting experiences (such as micro-teaching) in the context of applied courses can be possible with the peer study tool. With this tool, group members can score their contributions to a project or product (micro-teaching video) prepared by the group (Fig. 11).

Fig. 11

Peer work in moodle (Peerwork Activity—MoodleDocs, 2020)

In addition to these data essay based clinical case, interpretations can also be used in medical education. The handicap in this type of assessment activity is that it requires manual grading. If there is a large group of students, more than one evaluator may take part in the process. Assessments can be made by the instructors with the help of a rubric containing assessment criteria. It is also possible to give detailed feedback to the students in this activity (Fig. 12).

Fig. 12

Essay exam grading (Essay question type—MoodleDocs, 2021; Grading Essay Questions in Moodle Quizzes—UP Moodle Guides, 2020)

3.3 Case 3: Online Assessment Methods and Techniques that Can Be Used in the Field of Legal Education

Like all other discipline areas, legal education is also affected by digitalization. While legal education does not consist of psychomotor abilities, it requires decision-making abilities. The use of simulations (Barton, & Maharg, 2006), asynchronous videos (Lacey, 2021), and online learning (Dutton et al., 2019) are not new methods in the legal education process. Engagement is one of the important elements in legal education, where case studies and decision-making processes are examined intensively (Gerken, 2021). It is important for legal education to include case studies of different qualities that will increase engagement during the course process. In course design, a more text-based design can be preferred. In addition, the preparation of course and assessment designs that prioritize reasoning, decision-making, and effective communication skills will be more productive for this field. In assessment processes, more interactive processes beyond the traditional assessment approach are recommended in the field of legal education (Dutton et al., 2019). Such an assessment design process can be used to create a pool of questions that are formed at the level of knowledge and understanding covering laws and regulations and discussion and collaboration activities to enable learning at the decision-making and reasoning level. Assessing the scores obtained from quiz activities created for each subject, calculating the overall performance score by weighting the quiz scores according to subject density, and case presentations or oral evaluations in the context of designing simultaneous activities in virtual classrooms are other assessment tools that can be used in such a course design (Fig. 13).

Fig. 13

Using a question bank and weighting the assessment activity (Gradebook—MoodleDocs, 2013; Question bank—MoodleDocs, 2021)

A content-oriented design may also be possible in the field of law, as in the field of education. Documents containing laws and regulations in the field of law can be shared on the course page and discussions can be made on some sample cases. For example, a case of conflict between public and legal entities in the field of administrative law can be given as an exemplary case. According to the shared regulations, students can be expected to discuss the dispute by basing their claims on the regulations. In such a design, the online assessment process can be carried out through assignments enriched with learning analytics (Fig. 14).

Fig. 14

Learning analytics enriched assignment in moodle (Learning Analytics Enriched Rubric—MoodleDocs, 2021)

According to Fig. 14, students are expected to review the presentation in the virtual lesson, as well as to participate in discussions and interact with other participants in the discussion, in a one-week period. The assessment of the week can be made by the system by following the participant's performance this week according to predetermined criteria. For example, if a student views weekly content once, comments once on the discussion board, and interacts with only one different person, he or she will have received 9 out of 15 total points. Therefore, an online assessment process designed in this way facilitates the calculation of the overall performance and provides appropriate opportunities for process assessment.

4 Conclusion and Implications

Assessment is an important component that certifies the achievement of competency at the end of the program, motivates the student, and determines the level of learning outcomes. It is considered necessary for a successful assessment that this important process is carried out in an integrated manner with the instructional design (Herron & Wright, 2006; Reeves, 2000). Therefore, an assessment process independent of the content cannot be considered. The American Association of Higher Education (AAHE) described assessment using nine principles that emphasize both the outcome and the experience during the process (Reeves, 2000). This emphasis on the evaluation of the process points to the effectiveness of formative assessment. In online learning environments in which stakeholders are not physically together in particular, the design of the assessment process should be considered as an ongoing process and planned accordingly. For this reason, formative assessment offers a significant advantage to instructors in the online environment, which is to give timely and effective feedback to individuals who are already far away and to prevent possible conceptual mistakes or mislearning. In the absence of such assessment, compensations made as the result of an evaluation at the end of the learning process may be undertaken too late to positively affect learning outcomes (Bhagat & Spector, 2017).

In this book chapter, sample situations as suggestions for different learning areas are specified and instructional design and assessment designs are shared accordingly. Of course, these examples can be diversified, supported by different tools, or integrated into other areas. However, the common point of these shared cases is that they use assessment methods related to the activities carried out in the process, apart from classical assessment methods. In this approach, which provides feedback on learning processes, students’ skills and performances can be assessed with feedback produced in the system (Guerrero-Roldán & Noguera, 2018). Although the feedback process is very important in assessment, it remains less commonly used because it requires more time and effort compared to classical evaluation methods (Farrell & Rushby, 2016; Paul & Jefferson, 2019; Peytcheva-Forsyth & Aleksieva, 2021). In online learning processes in which there is no physical and temporal synchronization, the use of feedback increases engagement levels of the students (Martin & Bolliger, 2018) and, accordingly, student satisfaction (Li et al., 2016; Peytcheva-Forsyth & Aleksieva, 2021; Yuliang, 2012), as well as decreases dropout rates (Ivankova & Stick, 2006; Lee & Choi, 2011; Xiong, & Suen, 2018). Considering the sample cases presented here, it can be said that the assessment tools used provide indirect feedback for both the teacher and the student. However, depending on the course design and the teacher's experience, such tools can also be used to provide direct feedback with simultaneous sessions.

It is expected that the effectiveness of the online learning process will increase when the feedback mechanism is presented as real-time data over an LMS using learning analytics. It is very important for the assessment process to utilize data obtained from the interaction of individuals with the system and the content of the learning process (Crisp, 2020). The circular and iterative use of the data to be obtained through feedback and systems in the online learning process will increase efficiency for both students and instructors.

While the positive contribution of an assessment design prepared in harmony with the learning contents of the process cannot be denied, it is quite possible to encounter situations such as reliability or dishonesty that may reduce the effectiveness of this process. Authentic and personalized assessment activities also can prevent cheating or plagiarism (Arnold, 2016; Gikandi et al., 2011). With proctoring or plagiarism software, cheating problems in formative assessment can be partially, if not completely, controlled. Continuous information sharing also can be valuable for learners. Even if they gain in the short term, there are studies showing that informing learners about the learning losses they will experience in the long term is effective in preventing cheating (Arnold, 2016). In addition, if the process is completely online, face-to-face virtual sessions can be considered as oral exams, while if the learning process is blended face-to-face assessment options should be considered.

The weighting of the activities used in the assessment process on the overall grading method is also an important issue in this process. Even if instructors can determine the distribution scores, institutional rules and regulations also play a decisive role in this assessment process. In this context, weighting within the framework of the flexibility presented and providing the necessary information will also make the online assessment process more powerful.

Although the online assessment process is complex and its multidimensional nature requires effort, it can be carried out effectively with the integration of the right tools. The course designs and tools shared here can be diversified or used in different ways, and similar designs can be used in other disciplines not exemplified here.