1 Introduction

Institutions of Higher Learning (IHLs) are always seeking mechanisms that can be used for enhancing teaching and learning processes. Such enhancements could be achieved by gathering feedback from students regarding their classes (Khan and Khan 2019). Feedback is crucial for understanding the patterns of students’ opinions that could effectively improve teaching performance and for creating teaching plans (Dhanalakshmi et al. 2016). According to Aung and Myo (2017), feedback from students is vital for measuring the quality of teaching delivered by their lecturers. Feedback is also important for providing instant insight into the level of students’ understanding during their learning process (Abdulla 2018). Thus, it is imperative that students’ feedback is addressed by IHLs to successfully increase their academic achievements.

One approach to obtain students’ feedback is through the teaching assessment system, which includes their perspective on teaching instructions, their learning environment, and the quality of the lessons they learned (Balachanran and Kirupananda 2017). Such system would include quantitative and qualitative questions. The quantitative part would be composed of closed-ended questions, such as multiple choices, while the qualitative part would include open-ended questions, such as comments and suggestions from students in textual form. Previous studies had focused on quantitative questions that rely on fixed rubric rules compared to analysing feedback through qualitative questions (Balahadia et al. 2016). Although faculties often face difficulties in making sense of the feedback from qualitative questions, such feedback is rich with personal opinions, feelings, beliefs, and desires (Aung and Myo 2017; Balahadia et al. 2016).

Despite a variety of methods and techniques are available, this study applied the opinion mining technique for its ability to track textual reviews (Aung and Myo 2017). Previous studies have also reported that the lexicon-based approach is the most appropriate approach for analysing students’ feedback (Aung and Myo 2017; Nasim et al. 2017; Nitin et al. 2015). Therefore, this study aims to develop a new student feedback analysis system (hereafter known as the OMFeedback system) using lexicon-based approach. This system will also incorporate the capitalisation of words (Hutto and Gilbert 2014) and emojis (Shiha and Ayvaz 2017), which are actively used in most web-based systems.

2 Literature review

Opinion mining, also known as sentiment analysis, is an advancement in the area of text mining, which is primarily utilised to determine the opinions of people from big datasets involving unstructured texts (Dhanalakshmi et al. 2016). It is a common textual data quantification method that can analyse the sentiment tendency of a textual feedback. According to Song et al. (2019), human opinions are usually based on emotion, which can be classified as positive (e.g., joy and trust), negative (e.g., anger, fear, sadness, and disgust), and neutral (e.g., surprise and anticipation).

In the educational context, positive emotion is believed to have significant effect on students’ behaviour, while negative emotion could impact their learning behaviour (Binali et al. 2009). Several studies have discussed the application of opinion mining for analysing students’ emotion. For instance, Binali et al. (2009) proposed a conceptual emotion detection system, which uses GATE’s visual environment for developing, implementing, and testing language processing modules. Opinion mining was also used by Aung and Myo (2017), Balahadia et al. (2016), Barron-Estrada et al. (2017), Dhanalakshmi et al. (2016), and Nasim et al. (2017) for detecting emotion through students’ feedback on lecturers and courses.

The remarkable use of opinion mining for analysing feedback from students has led to the development of a plethora of techniques. Basically, opinion mining can be classified into machine learning and lexicon-based. Machine learning focuses on building models with the aid of large training datasets to determine text orientation (Soong et al. 2019). Meanwhile, the lexicon-based approach uses sentiment dictionary with opinion words and matches them with data to determine polarity (Aung and Myo 2017). This approach will assign sentiment scores to the opinion words describing how positive, negative, or neutral the words are, as found in the Vader Lexicon dictionary. Positive opinion words are used to express necessary things, negative opinion words are used to describe unnecessary things, and neutral opinion words are used for a better distinction between positive and negative words.

Various machine learning techniques have been utilised to analyse students’ feedback, such as Naïve Bayes, support vector machine, neural network, and k-nearest neighbour. According to Dhanalakshmi et al. (2016), Naïve Bayes is the most commonly used technique to calculate the possibility of a given text belonging to a particular feature. Support vector machine works best for classifying sparse text data. Neural network employs multiple layers of neurons for text classification and k-nearest neighbour uses Euclidean distance to calculate the similarity of text data. Several studies found that neural network is the perfect technique for opinion mining as it is capable of analysing a large number of text data (Balahadia et al. 2016; Dhanalakshmi et al. 2016; Pong-Inwong and Kaewmak 2016; Tseng et al. 2018). However, this study has identified that the neural network technique often requires high computation power to train the dataset to generate accurate results.

Meanwhile, Aung and Myo (2017), and Nasim et al. (2017) found that the lexicon-based approach neither needs a large number of text data nor high computation power for producing accurate results. This is because lexicon determines the polarity of a word by using the constructed dictionaries. These dictionaries would contain sentiment scores that will be assigned to the opinion text to form the final expression of emotion (positive, negative or neutral). Previous studies on opinion mining have also shown that the capitalisation of words is crucial to emphasise the user’s intent (Hutto and Gilbert 2014), while the use of emoji characters may enhance the expressivity of the feedback (Shiha and Ayvaz 2017). However, only a small number of studies have utilised these features to enrich the ability of the opinion mining technique for analysing students’ feedback. Based on the above reasons, this study has developed the OMFeedback system, and incorporated the capitalisation of words and emoji characters to the lexicon-based approach to analyse students’ feedback.

3 System architecture

This study has designed the OMFeedback system through the Unified Modelling Language (UML), as depicted in Fig. 1. This system comprises of two main users, namely, the students and the admin. Both sets of users need to log in to the system using their username and password. Students must select a lecturer’s name before writing their feedback for the teaching assessment in the space provided. The completed feedback is stored in the database and can only be viewed by the admin. This system uses the Vader Sentiment Intensity Analyser to analyse each word in the feedback and it will calculate the score based on the value assigned in the Vader Lexicon. The scores are categorised as positive, negative, and neutral to represent the overall opinion of the students towards their lecturer’s teaching performance. Finally, the results of these scores can be viewed by the admin for further actions.

Fig. 1
figure 1

OMFeedback system architecture

Full size image

4 System development

The OMFeedback system has been developed through the Python 3.7, while the WxPython package was used to develop the interface of the system. Algorithm 1 and Algorithm 2 are the login algorithms used by students and administrators, respectively, to access the OMFeedback system. Students would only need to enter their matrix number and password. Data entered to the login space will be stored in the database as a student entry record. Administrators would need to enter their username and password to access this system to view students’ feedback on a lecturer’s teaching performance.

Algorithm 1. Student’s login algorithm for the OMFeedback system.

Input:

Matric Number matric;

Password password;

Output

Run next process

START

1. GET matric;

2. GET password;

3. IF matric AND password == TRUE

4. Run next process

5. ELSE

6. Show error message

Algorithm 2. Admin’s login algorithm for the OMFeedback system.

Input:

User name username;

Password password;

Output

Run next process

START

1. GET username;

2. GET password;

3. IF username AND password == TRUE

4. Run next process

5. ELSE

6. Show error message

Algorithm 3 shows the process of entering a lecturer’s information and feedback in the form of opinion texts by students regarding the lecturer’s teaching performance. Students would need to choose the name of the lecturer from a list and type their feedback in the space provided. This feedback text will be analysed by the Vader Sentiment Intensity Analyser, which will assign positive, negative, and neutral scores. The lecturer’s name, the feedback text, and the score will then be stored in the database

Algorithm 3. Student’s textual feedback analysis through the Vader Sentiment Intensity Analyser.

Input:

Lecturer’s name name;

Feedback feedback;

Output:

Sentiment score score;

START

1. GET name;

2. GET feedback;

3. def sentiment_analyzer_scores(feedback):

score = analyser.polarity_scores(feedback)

print(“{:- < 40} {}”.format(feedback, str(score)))

4. Send variable [name, subject, group, feedback,score] to database

5. END

Algorithm 4 shows the algorithm used by the administrator to view the results of the lecturer’s teaching evaluation. Once an administrator logs into the OMFeedback system, the administrator must choose the name of the lecturer to read students’ opinion texts on the selected lecturer. Based on the name of the selected lecturer, the OMFeedback system will perform a search in the lecturer database. Next, the sentiment score (positive, negative, and neutral), as analysed by the Vader Sentiment Intensity Analyser, will be displayed to the administrator in the form of pie charts for better comprehension

Algorithm 4. Lecturer’s performance result.

Input:

Lecturer’s name name;

Output:

Lecturer’s performance performance

START

1. GET name

2. GET group

3. Search in database data that containing variable [name, group]

4. Present score analysis in pie chart

5. END

5 Experiment

This study collected feedback from 120 third-year students in the Computer Science course during Semester I of the 2019/2020 session at the National Defence University of Malaysia. The following Fig. 2 shows the feedback form that the students need to complete at the end of the semester.

Fig. 2
figure 2

The feedback form in the OMFeedback system

Full size image

This paper presents an example of students’ feedback on a selected lecturer. As shown in Table 1, the students’ feedback shows different scores for positive, negative, and neutral polarities. These scores were analysed using the Vader Sentiment Intensity Analyser and each of the textual feedback has its own fixed values, which are stored in the Vader Lexicon. As previously mentioned, this study is introducing two new features, namely, the capitalised words and emoji characters in the OMFeedback system. Based on 120 third-year students, 15 of them used these new features to emphasise and better express their text feedback, while other students used the common textual style for expressing their opinion. It was interesting to find that when both the capitalisation of words and emojis were used simultaneously in a sentence, a high negative score (79.4%) was obtained.

Table 1 List of students’ textual feedback
Full size table

Figure 3 shows a pie chart representation of the students’ feedback according to the aforementioned polarities. Based on the feedback from 120 students, 49.3% have positive opinions towards their lecturer’s teaching performance. Neutral opinions were at 49.2% and only 1.5% were for negative opinions. These results indicated that most of these students have positive and neutral opinions about their lecturer’s teaching.

Fig. 3
figure 3

Results of students’ feedback on a lecturer

Full size image

6 Conclusion

This study has developed the OMFeedback system that incorporates the opinion mining technique for analysing students’ feedback in the textual format. While most opinion mining studies rely on machine learning techniques, particularly the neural network, the lexicon-based approach, with capitalisation of words and emoji characters to improve the textual feedback, is often ignored. By utilising the lexicon-based approach, with the aforementioned new features, lecturers’ teaching performance could be better analysed. This improved lexicon-based approach could shed some light into the text mining research area, thus, opening up a new avenue for other research directions.

Although this study has provided significant evidence that qualitative analysis (textual feedback) will complement the quantitative analysis (fixed-rubric rules) of students’ feedback, the OMfeedback system still needs improvements to better understand the complexities of students’ opinions. Therefore, future studies are encouraged to add more words in the lexicon dictionary to enhance the classification accuracy. This system should also be added with a spelling and grammar checker for verifying each feedback. In addition, IHLs are recommended to facilitate the study of this type of system to increase the effectiveness of the lecturer teaching assessment system.