Keywords

1 Introduction

An increasing number of universities are using computing technologies to enhance the process of teaching and learning in order to meet the needs of diverse learners. Interactive systems and internet technology allow effective distribution and delivery of educational resources to students. This gives students the opportunity to learn at their own pace and convenience. Despite the usefulness of these systems, students find it hard to engage for a long time with learning resources. There are many distractions which compete for students’ attention, such as chatting, playing games, listening to music, watching videos, etc. Thus, a wide gap exists in academic performance between successful and unsuccessful students (those that drop out of universities). Therefore, there is a need for research on how to increase students’ motivation to learn and engage actively in learning activities. The level of motivation of students to learn and progress in their education determine the length of time they spend on learning-related activities.

Reading, understanding and remembering various learning materials in the quest for knowledge can be tedious and monotonous. Students make plans on how to succeed in their learning activities but find it difficult to motivate themselves to stick to their plans. Thus, this research investigates the use of persuasive technology (PT) in promoting students’ learning activities to improve their academic performance. PT describes technological applications and software purposely designed to change users’ attitudes or actions without using coercion or deception [5]. It achieves behaviour change through the use of various techniques (strategies) that promote a positive change of behaviour or attitude. The success of PT applications in encouraging users to adopt desirable behaviour has been established in various domains. For example, the ability of persuasive systems built on socially-oriented strategies to inspire people to achieve their goals has been established in e-commerce [16] and health [14]. This suggests that strategically designed PTs using social influence constructs can motivate for a desirable change of attitude or behaviour in other domains, for example, increasing engagement in learning activities.

Social influence persuasive strategies are a good candidate in this case because a wide gap exists in academic performance between successful students and unsuccessful ones (those that fail) in universities. To bridge this gap, there is a need to create performance awareness among students offering a course. It will help the students to measure and understand their academic progress in relation to their peers. Besides, it will encourage the students to improve their learning activities because according to the social influence theory [15], individual behaviours and actions are often influenced by those of other people. Therefore, this research explores the use of social influence strategies (upward social comparison, social learning, and competition) implemented in PT in motivating students for learning activities. The three strategies are operationalized as different versions of a social visualization in a persuasive system used by students. The effect of each individual strategy on students’ learning activities is established in a controlled study.

This work has the following contributions: Firstly, we show that the three strategies can be implemented in a persuasive system and applied in a university setting without jeopardizing students’ privacy and security. Secondly, we demonstrate how to make implementation of different strategies in a persuasive system easier. We implemented three strategies as three versions of a persuasive system. Next, this research establishes that the system versions which implemented the strategies are effective in motivating students for learning activities. This means that implementing one suitable strategy for a desirable goal will be effective to motivate users to achieve a target goal. Lastly, we show that personalizing the persuasive system versions by tailoring the strategies employed in their design makes them more effective.

2 Background and Related Work

The application area of persuasive technologies has been growing rapidly over the recent years [6, 10, 13]. The driving force for this has been its potential to intentionally change users’ opinion and action towards a desired goal. The success of PT applications is based on the use of appropriate strategies for users that target a specific behaviour change domain and goal. In this research, we investigate the suitability of three social influence strategies of PT at changing students’ learning behaviour positively.

According to Kelman’ social influence theory [15], thoughts, attitudes, and behaviours of an individual are influenced by that of other people. He postulated that changes in behaviour and attitude are a result of social influence and are brought about by three processes: compliance, identification, and internalization.

Compliance - the individual changes to the desired behaviour to get a reward or evade chastisement.

Identification - adopting to the target behaviour or attitude is as a result of the individual trying to sustain his relationship to other people (conformity).

Internalization - the individual decides and accepts to change her belief and activities to that of other people because she thinks the change will be beneficial to her.

Hence, Kelman suggested that the processes used in implementing social influence cause differences in the level of changes in behaviour among individuals. In line with this, Fogg proposes that computers can act as a behaviour change support agent as they can influence users through the services built on them. And can manipulate different influence strategies for different users and still persist the influence as long as is needed. Based on Fogg’s work [5], Oinas-Kukkonen et al. [9] established a design model called Persuasive Systems Design (PSD) model which describes the development and evaluation process for persuasive systems. Among Oinas-Kukkonen et al. [9] persuasive strategies are the social influence strategies. Social influence strategies change people’s opinion or attitude by using other people who are performing the desired behaviour as a role model for the target behaviour change. This research investigated social comparison, social learning, and competition of the PSD model.

The Social Comparison strategy offers users the opportunity to view and compare their behaviour performance data with that of other user(s). The direction of social comparison could be upward or downward. The upward social comparison is normally used for self-improvement as people are motivated to improve in behaviour or task performance by comparing themselves to similar others who are performing well (or better than themselves) on the specified task. Social comparison in this research refers to an upward comparison; research [4] has established that students use upward comparison when comparing their performance. The Social Learning strategy assumes that people learn through observation, modelling and imitation of others performing the intended behaviour. It points to what many similar others have done or what they are already doing to induce observational learning. According to Bandura [1], observational learning can be achieved by watching an actual performance of a task, reading or visualizing behaviour performance description, and symbolic demonstration of behaviour performance. The Competition Strategy provides opportunities for users to compete with each other; getting ahead of others motivates them to perform the desired behaviour.

Research has established the efficacy of social influence strategies of PTs in motivating people to achieve certain goals in various domains [12, 16]. Christy and Fox [3] investigated the effects of social influence strategy (Social Comparison) on students’ academic performance in a virtual classroom. They reveal that social comparison can influence women academic performance in Math. Stibe et al. [16] explored the use of social influence strategies: social comparison, social learning, normative influence, social facilitation, cooperation, competition, and recognition in encouraging customers to generate and share feedbacks. Based on the results of their analysis of the influence strategies, they indicate that the strategies motivated customers to improve the rate at which they generate and share feedbacks. Orji et al. [12] examined the influence of competition, social comparison, and cooperation in the health domain.

Based on our literature search, research has not shown how the three influence strategies can affect students’ learning activities in a real university setting. However, the effectiveness of the strategies at encouraging users to achieve a desired goal in other domain has been demonstrated.

3 Study Design and Methods

Our study aims to investigate the persuasiveness of three versions of a persuasive system designed with social comparison, social learning, and competition in motivating students’ learning activities in a real university course-based setting. We intend to answer the following research questions:

RQ 1: :

How do the students perceive the three versions of the persuasive system?

RQ 2: :

Is there a difference in the perceived persuasiveness of the three system versions overall?

RQ 3: :

Does tailoring the persuasive system increase the perceived persuasiveness of the system?

To successfully implement a persuasive system and answer our research questions, we first determine the suitability of the strategies for our user group.

3.1 Determining Users’ Susceptibility to the Three Social Influence Strategies

Determining the applicability of PT strategies to a particular user group is an important step prior to PT design. Hence, implementing appropriate strategies in PT design increases its efficacy to achieve the intended objective. We examined the susceptibility of our user group (Biol 120 students) to social comparison, social learning, and competition. According to existing research [2], understanding users’ preferences for PT strategies assist designers in making informed decisions on the requirements and implications of their design. Some of the decisions are to determine whether specific strategies will be effective in motivating a particular user group for a task, and how to personalize PTs built with the strategies to users.

We used a tool developed by Busch et al. [2] for measuring susceptibility to social influence strategies called persuasive inventory (PI). A questionnaire implementing the PI was slightly adapted to reflect the target domain, education. All questions were assessed using the participants’ agreement to a 9-Likert scale ranging from “1 = Strongly Disagree” to “9 = Strongly Agree”. According to Busch et al. [2], the persuadability inventory gives an estimation of people’s susceptibility to a specific persuasive strategy which designers of persuasive technology can use in identifying the most effective persuasive strategy to use in designing PT for a particular user or user group.

The total number of participants was n = 220. The reliability test for participants’ responses is α = 0.817 and KMO sampling adequacy is 0.858 which means that the responses were reliable. Our repeated-measure ANOVA results show significant main effect of strategy type (F1.63, 355.54 = 22.04, p < .0001) on persuasiveness and pairwise comparison reveal that a significant difference exists between the persuasiveness of competition (M = 5.615) and social learning (M = 5.029) and also between social comparison (M = 5.560) and social learning, p < 0.05. There was no significant difference between the persuasiveness of competition and social comparison.

Based on the result of our analysis, all the strategies were perceived as persuasive, as each strategy has a mean rating which is greater than the neutral score of 4.5 (p < .001). Table 1 shows the susceptibility of the participants to the three strategies. According to the table, the majority (88%) of the students could be persuaded using the three social influence strategies of PT.

Table 1. Susceptibility of the participants to the three social influence strategies of PT
Full size table

The results from the analysis demonstrate that the strategies are effective tools which can be employed to influence students’ learning behaviour positively. Hence, most of the students rated some of the strategies as persuasive. It suggests that implementation of the strategies in persuasive applications will encourage students to improve their learning behaviour. In general, there is no significant effect of gender on the persuasiveness of strategies by the students. This implies that educational systems designed with these strategies will create the same persuasive effect in both male and female students. Therefore, in creating the students’ persuasion profiles, we did not consider the gender of the student, but only considered the student’s susceptibility to the three strategies. Following Busch et al. [13], “participants having higher scores in one or more of the scales are expected to be more susceptible to these specific persuasive strategies (p. 36).” However, some students are susceptible to all the three strategies, as shown in Table 1. This means that any of the three strategies can motivate them to achieve a specific goal. The level of motivation each strategy provides depends on the participant’s preference for that strategy. Hence, we considered participants’ highest preference for any of the strategies in their persuasion profile. According to their highest preference, 38% of the students had competition as their highest preferred strategy, 30% had social comparison, 20% had social learning, and 12% were not susceptible to any of the strategies. This result indicates that the preference for competition (38%) is the highest, followed by social comparison (30%), and social learning (20%) is the least.

Having established the appropriateness of the strategies for the students using this study, we moved to operationalize the strategies in an actual persuasive system to evaluate their effectiveness at motivating students to improve in learning activities.

3.2 Persuasive Intervention Experiment

We developed a web application for our persuasive system and operationalized each of the strategies as a system version in our application. In most PT designs, strategies are achieved as a design goal or based on system usage. For example, Stibe et al. [16] implemented the three strategies in their visualization. To facilitate social comparison, they display the number of tweets each user submitted. The number of tweets for each user changes colour as it increases to make comparison easy. For the competition, they ranked users based on their number of tweets. To allow for social learning, they displayed newsfeed from users so that others can observe and learn.

We considered the issue of security and students’ privacy as we used individual students’ information to develop the application for social comparison and competition. Social learning also uses students’ information but in an aggregated form. The application was integrated with the learning management system (LMS) which the students access for most of their course information needs to make it easier for them to use the application. Students log in to the LMS with their student identification number (Id). To solve the privacy problem, we used a pseudonymized student Id to display students’ grades and points except for the logged-in student. For the logged-in student (who views the visualization), the student’s actual Id and name are shown so that he or she can identify his or her position in comparison with the others. Each version depicts one of the strategies using a persuasive visual display (visualization), as shown in Figs. 1, 2, and 3.

Fig. 1.
figure 1

A display of the logged-in student’s grades and grades of five random students with anonymized id who have higher grades than the target student (upward social comparison)

Full size image
Fig. 2.
figure 2

A display of grade ranges and the number of students that has each range in a course (social learning)

Full size image
Fig. 3.
figure 3

A display of students’ ranks based on their performance (competition)

Full size image

The visualization (in all three versions) updates dynamically when students perform new assessments and provides students with an opportunity to send feedback expressing their feeling about their grade by clicking on an emoticon, shown in Figs. 1, 2, and 3. The visualizations allow the students to view their class performance in a course so that they can compare, compete, or model their behaviour. This aligns with previous research showing that human actions and attitude could be influenced by that of others. People can change their behaviour to adopt or imitate the behaviour of other people which they think will be beneficial to them.

3.3 Measurement Instruments and Data Collection

Each student was assigned a version of the application. The persuasion profile of students who participated in the PI survey was used to tailor the versions to them. Students that did not participate in the PI survey described in Sect. 3.1 were randomly assigned. The students used the system for six (6) weeks.

To elicit feedback on the persuasiveness of the system versions in our application, we employed a validated tool for assessing the perceived persuasiveness of applications. The tool was adapted from Orji et al. [11] and other PT research works [13, 15] have used it. The tool consists of four questions: (1) “The system would influence me.” (2) “The system would be convincing.” (3) “The system would be personally relevant for me.” (4) “The system would make me reconsider my study habits.” The questions were measured using participant agreement with a 7-point Likert scale ranging from “1 = Strongly disagree” to “7 = Strongly agree”. We designed a system exit survey with the questions and conducted the survey among the students after they have used our persuasive system for six weeks.

Data Collection

Participants for this study were undergraduate students of the University of Saskatchewan taking Biol 120 during the winter term 2018. All the participants (students) were at least 16 years old. Before the main study, we conducted a pilot study to test the validity of our persuasive system design. For the pilot study, we recruited nine random students from the same university and they used the system versions. We ascertain that our system versions were persuasive based on their feedback. For our main persuasive system experiment, a total of 643 students taking Biol 120 participated in the intervention. We received a total of 266 responses from our system exit survey conducted among the students that used the system. Among the 266 students that responded, 228 agreed that we should use their data for analysis. Among the 228 participants, 96 used a tailored version of the system, 11 used the contra-tailored version (i.e. the version based in the strategy they were least susceptible to), 35 were in the control group that didn’t use or rate the system, and 86 were randomly assigned to the three different versions. The contra-tailored group was too small and therefore was not involved in the analysis. In summary, the sizes of the groups subjected to the analysis were as follows: competition – 21 students, social comparison – 105 students, and social learning – 67 students.

4 Data Analysis

To measure the persuasiveness of the three versions of the persuasive system and evaluate the effect of the tailored compared to the random assignment of students to versions, we employed some well-known analytical techniques and procedures. The following steps were followed to analyze the data.

  1. 1.

    Kaiser-Meyer-Olkin (KMO) sampling adequacies and the Bartlett Test of Sphericity were used to determine the suitability of the data for analysis.

  2. 2.

    After establishing the suitability of the data, we conducted a one-sample t-test on the data measuring the persuasiveness of each persuasive system version separately to establish their individual persuasiveness overall.

  3. 3.

    Next, to examine and compare the persuasiveness of the three system versions, we computed the average persuasiveness score for each strategy and performed a One-Way ANOVA after validating for ANOVA assumptions.

  4. 4.

    Finally, to compare the efficacy of the tailored and random assignment intervention types with respect to their ability to promote learning activities among students, we conducted an independent sample t-test.

The detailed results of the analysis are presented below.

5 Results of System Perceived Persuasiveness

The Kaiser-Meyer-Olkin (KMO) sampling adequacy was 0.764 and the Bartlett Test of Sphericity was statistically significant (χ2 (6) = 548.12, p < 0.0001). These results show that the data were suitable for further analysis [8].

5.1 The Persuasiveness of the Persuasive System Versions

Each version of the persuasive system was used by different groups of students and each group rate the version that they used.

In 7-point Likert scale, system persuasiveness score above 3.5 (median score of scale) is categorized as high. To determine if the persuasiveness of the system versions is high, each version persuasive score is compared to the scale median score. From the results of the one-sample t-test examining the persuasiveness of each system version using a confidence interval of 95, we established that the three persuasive system versions representing social comparison, social learning, and competition were rated as significantly persuasive with persuasiveness score higher than the neutral value (median rating) of 3.5 as shown Fig. 4, social comparison (M = 4.64, SD = 1.42, t104 = 7.61, p = .0001), social learning (M = 4.39, SD = 1.64, t66 = 3.77, p = .0001), and competition (M = 4.28, SD = 1.35, t20 = 2.38, p = 0.03). Overall, the system implementations of the three strategies were perceived as persuasive by the students.

Fig. 4.
figure 4

A bar graph of the mean of the individual strategies showing their overall persuasiveness

Full size image

5.2 Comparison of the Persuasiveness of the Three Persuasive System Versions

The results of one-way ANOVA show that there was no statistically significant difference between the three persuasive system versions with respect to their persuasiveness (F2,190 = 0.711, p = .493). This result indicates that the perceived persuasiveness of the three system versions did not differ significantly among the experimental condition (social comparison, social learning, and competition) of students even though they used different system versions.

5.3 Comparison of the Persuasiveness of the Tailored and Non-tailored Group

The independent sample t-test results between tailored and non-tailored students’ groups show a statistically significant difference in the persuasiveness of the system, t132.74 = 2.66, p = .009. Specifically, the students in the tailored group that used their preferred persuasive system version rated the system as more persuasive than the students that were randomly assigned to use any of the system versions without considering their strategy preference.

6 Summary and Discussion

The results of our study of students’ susceptibility to the strategies demonstrated that students can be motivated by all of the three social influence strategies and that the preferences of students to the strategies differ. Most students are motivated by competition, followed by social comparison and then social learning. Based on this we developed persuasion profiles for students that we used for tailoring a persuasive application.

To validate the results of the susceptibility study, we developed three versions of a persuasive visualization system to encourage students to engage in learning activities, using social comparison, social learning, and competition. The versions were tailored to some of the students that participated in our susceptibility study, while the rest of the students were randomly assigned to versions without considering their strategy preference. Our results reveal that tailoring persuasive system using students’ persuasion profile will improve the efficacy of the system to promote a desired learning behaviour of students. Below we discuss how these results answer the three research questions formulated in Sect. 3.

6.1 Social Comparison, Social Learning, and Competition of PT

The findings in this research show that socially-oriented PT strategies (upward social comparison, social learning, and competition) can effectively be applied in university education to promote desirable learning behaviour among students. Although the three strategies differ in their operationalization in the system design, students acknowledged their potential in promoting learning behaviour (engagement) overall. Based on the system evaluation results, all the students that used the system perceived as persuasive the implementation of the three strategies with respect to their ability to motivate students to engage in their learning activities. Thus, the research question RQ1 has been answered by showing that persuasive visualizations designed based on socially-oriented persuasive strategies (upward social comparison, social learning, and competition) are perceived by students as promoting learning and engagement.

6.2 The Persuasiveness of the Different System Versions

Without considering tailoring, our results indicate that the three versions of the persuasive system do not differ significantly in their perceived persuasiveness, suggesting that the strategies are not fundamentally different in their effectiveness overall. Thus, RQ2 has been answered. However, tailoring the system versions to the individual susceptibility of the students showed a difference in their persuasiveness. This reveals that tailoring the persuasive system to students using their persuasion profile makes them perceive it as more persuasive as shown by the higher rating of the system persuasiveness by students in the tailored condition. This answers the third research question, RQ3, showing that the effectiveness of the strategies in education software can improve, if students receive tailored versions of the system that match their persuasion strategy preference.

6.3 Timing of Persuasive Intervention

It is hard in the education domain to manage the timing for persuasion so that it catches students’ attention without causing a distraction to their learning. This work shows one possible way to achieve this. The persuasive system was integrated into a learning management system through which students access their course information. Our results demonstrate the success of this approach at apprehending and directing students’ attention to the persuasive information, thereby making students reflect on their learning progress in general. Feedback from students to the system supports this conclusion, for example: “I should be doing better, its just a poor effort on my part”, “I don’t know how to study”, and “I usually do better, and I know I can, but I just don’t have the time”.

6.4 Design Implications of Our Study

The common practice in the design of persuasive systems is to incorporate multiple strategies in a single system. In this way, at least one of the strategies would be able to motivate some users. However, this approach makes it hard to evaluate the persuasiveness of the individual strategies and to improve the overall persuasiveness of the system. Our approach of applying different strategies, tailored to different user groups allows the evaluation of the persuasiveness of each deployed strategy.

Our evaluation results reveal that the use of a single strategy suitable for a particular user group is more effective at achieving the intended goal. Moreover, research [7] has shown that combining appropriate strategies in a single system may not increase the persuasion effect in motivating for the intended behaviour change. Thus, designers should aim to incorporate means to profile users according to their susceptibility to persuasive strategies and determine a single appropriate strategy for a particular user group for an intended goal.

Tailoring of persuasive systems to individual users can be costly if the designer has to develop different system version for each user. Our work has shown that specifically with persuasive visualizations and social influence strategies, this task is not so hard, as the three strategies can be implemented in a fairly straightforward way as tailored visualizations, generated from the same system data.

7 Conclusion

Previous research has established the efficacy of persuasive systems at encouraging users to achieve a specific objective in various domains. To contribute in advancing the field of persuasive technology research in the education domain, our work investigated the perceived persuasiveness of three strategies based on the Social Influence Theory in increasing students’ engagement in learning activities. Our study in a large first-year University biology class shows that these strategies can be implemented as persuasive visualizations that are perceived as motivating by students in engaging them in their learning activities. Moreover, it shows that tailoring can enhance the effect of each persuasive strategy. Our work can help designers of learning management systems by providing an example of how three social-influence-based persuasive strategies can be implemented in persuasive visualizations of learning analytics data, and suggesting tools that can be used to profile students to allow for personalization based on students’ persuasion preference.