Keywords

1 Introduction

Effective student support can be vital to progression rates and overall achievement of students [1, 2]. In the university environment, for a typical student, the support role does not rest with a single individual. A student may have a personal tutor, who is the first port of call for issues concerned with their academic progress, and who might refer the student to others to deal with non-academic issues. These others might include counsellors, placement officers, examination officers, accommodation officers etc., each of whom can assume the support role in their specific context. We investigated the desirability of automating this support role, with a view to providing higher quality interaction with university services at times when face-to-face support was not available, through the use of a Virtual Assistant (VA).

For the pilot study, we developed a prototype system to assist with one highly specific support role; that of the university employability service, which advises students on job applications, CV writing, interview technique etc. If successful, the intention was to roll out the system to other academic and non-academic support roles.

Typically, most university services are accessed in the first instance via web pages. There is usually a great volume of static information, together with a few dynamic pages for making appointments or similar. Students often have difficulty in finding the answer to a specific question when trying to navigate these pages [3, 4]. In today’s world, students need to access university services at any time of day, and person to person contact is not always possible. In these circumstances, it would clearly be desirable for there to be some kind of artificial agent that could conduct a similar dialogue to guide the student to the required information. A number of rule-based text-dialogue systems have been developed and it is possible to combine these with text-to-speech avatar-based interfaces to provide a facsimile of the ideal human-to-human dialogue. The benefits of using a VA include constant availability and the use of natural language communication to provide answers to queries or guide the student towards relevant information.

2 Related Work

Studies have been carried out by a number of different research groups on the use of intelligent virtual systems, some of which cover use in higher education for student support. Augusto, McNair, McCullagh and Roberts [5] explored the idea of providing virtual mentoring for students at the University of Ulster. Whilst the outcome was limited, the potential for further development was acknowledged. Intelligent agent based systems have also been piloted to support students in the learning of programming languages [6, 7].

The University of Granada [3] developed a virtual assistant for their university website, to provide help for users in finding “not only the information that they are looking for, but also some related information which could be of the highest interest”. Their argument was that users often waste time looking through websites when using traditional menu or keyboard methods. Their findings, based on six months use of a virtual assistant, were very promising, but the team acknowledged that further work was required, including addressing the issue of the virtual assistant’s facial expressions.

An avatar can represent a tutor or an instructor who will guide learners through the materials [8]. A study carried out by Mazlan and Bird [9] investigated whether the use of avatars in online learning settings could motivate students. The results of the study indicated that the “students expressed positive interest in avatars and their motivation to learn”.

Use of avatars in health-related work has been considered in a number of studies. Kim and Sundar [8] investigated whether the use of avatars could improve a person’s self-image. The findings indicated that for those with highly discrepant self-image, a virtual avatar could become ‘more of a motivating force’ and could have a positive influence on their behaviour in the real world.

Similar research on self-image by Peña and Kim [10] focused on avatars as part of ‘exergames’ (video games for exercise; virtual tennis in this case). The study experimented with manipulating the participants’ weight from slim to normal to obese to see if this had an effect on their physical performance as they played. They demonstrated that participants increased their physical output when both players’ weights were kept normal, but play slowed down and was ‘sluggish’ when they were made obese. They concluded that “traditional psychological processes can be put to the service of increasing physical exercising and hopefully improving people’s well-being through virtual experiences.”

The literature provides a number of different descriptions of what is understood by the term ‘agent’ or ‘avatar’. For example, [11] states “an agent is defined as an acting entity, which includes artificial intelligence that renders the control. An avatar, by contrast, is a virtual representation of a human being which is controlled completely by the human.” We could describe an avatar as a computerized representation of a character representing a user in another environment. Avatars are used extensively in many computer games, where an avatar character can be controlled by the user.

Studies have been conducted on how people react to agents and avatars, in an attempt to establish why users have a social reaction towards them regardless of the knowledge that they are conversing with a machine. [11, 12] investigated the nature of the changes in people’s communication when they interact with intelligent agents, as compared to human-to-human communication.

Cafaro et al. [13] found that “in first encounters people quickly form impressions of each other’s personality and interpersonal attitude.” They found that even when one of the participants was a virtual agent, it took an average of only 12.5 s for subjects to form an impression. In other words, they reacted to the agent in a similar manner as they would to another human.

According to Baylor [14], the nature of the agent interface does not have much direct impact on actual learning, but does have a positive effect on learner motivation and self-efficacy. Baylor’s study discusses the design of agents and the importance of the agent’s visual appearance as “the most important design feature as it dictates the learner’s perception of the agent as a virtual social model, in the Bandurian sense”. Another key feature proposed is “message delivery through a human-like voice with appropriate and relevant emotional expressions”.

The Microsoft Office agent ‘Clippit’ was introduced in the 1990s [3, 15]. This character was intended to act as the users’ personal assistant by speaking and acting on voice commands. It was generally seen as being too intrusive and rather ‘characterless’. Although users’ unfamiliarity with the technology at the time might have been a factor in this, it may be noted that a personal agent will fail if user expectations are not met.

Some rapid advances have been made in technology in recent years, but many challenges remain and one of these is the use of natural language with agents; this requires in-depth understanding of linguistics and semantics in order to create a truly intelligent system [3]. Siri, the voice recognition app and intelligent digital assistant from Apple, uses natural language to help users to find answers to their queries. While Siri’s launch caused much excitement, this soon deteriorated when Siri’s limited intelligence became apparent.

When an avatar, as part an intelligent system, can engage in conversation with a human being it can be described as an Embodied Conversational Agent (ECA). Cassell [16] states that “Embodied conversational agent interfaces are specifically conversational in their behaviors and specifically humanlike in the way they use their bodies in conversation”. As an example of an ECA, Cassell’s creation, REA (Real Estate Agent) acts as a salesperson interacting with clients and showing or selling them virtual properties. REA can also engage in ‘subtle human like conversation’ with clients at appropriate times and use such phrases as ‘I see’ to indicate agreement with a client or that she is listening. These may appear to be simple features, but human conversation is extremely challenging to model in a computer system and this system required lot of sensors and computational resources to function [3].

3 Research Methodology and Analysis of the Need for a VA

To establish the extent of the perceived need for a virtual assistant, we needed to gather quantitative data on the nature of students’ interaction with existing support staff, and also to target a particular university service to obtain the opinions of members of support staff as to the potential value of a virtual assistant system. We chose to use a questionnaire to gather the student data, as it would have taken considerable time to gather data by interviews and the qualitative nature of such data might have skewed the perception gained. For consistency and quantification, the questionnaire contained only closed questions. Aspects explored by the questionnaire included the frequency of contact, the nature of the students’ queries, the modes of contact, i.e. face to face, online etc. and the quality of the students’ interactions with services. We also sought students’ opinions as to whether they would be likely to use a VA for out-of-hours assistance.

The survey was carried out with 125 students from the School of Science and Technology at Middlesex University. The questionnaire was given to students at foundation level (FY), first year (Y1), second year (Y2) and third year (Y3) undergraduate level. Some of the findings were as follows.

The survey showed that students need to contact their tutors most often at FY and least often in Y2. Most students found they needed to contact their tutors frequently throughout the year. The majority of tutors responded to student queries within two days. The main reason for students contacting tutors was for academic issues, although 30 % of the Y1 students contacted their tutors for non-academic issues. The survey confirmed that students regularly accessed online support facilities.

Surprisingly, students had almost no interest in using contact via phone or social media, which contradicts the findings in [2]. Based on our survey, students currently prefer communicating with tutors face-to-face and by email.

A significant proportion of students at all levels stated that it was likely they would contact tutors outside teaching hours. There is a possibility that students have misinterpreted this question; we consider teaching hours to be normal working hours, whereas students may have interpreted this as time outside the classroom. However, the data did suggest that students value the opportunity to contact their tutors at any time.

The survey established that students require support on all aspect of their studies, from general administrative issues to pastoral and academic queries. Whilst students value face-to-face communication and support, they also now expect round the clock support [1]. Students responded very positively when asked how likely they would be to use an online tool such as a Virtual Support Assistant. Almost 80 % of students responded that they would be (quite likely to very likely) to use such a tool. It appears that students value direct contact with tutors (either face to face or by email) for academic matters, but would also value the additional support provided by a VA.

There is evidence [17] that students prefer information provided via the spoken word to written information. Of course, the best way to achieve this is by person-to-person communication. Recorded audio can provide the personal touch but no body language, gestures etc. Recorded video provides the element of body language, but neither audio nor video recordings allow the possibility of interaction to guide the user towards the desired results. An avatar-based system does not have the personal touch, but provides another dimension beyond plain text and facilitates a personalized dialogue.

4 Virtual Assistant Support System

The VA was designed to be a web based application (Fig. 1) that can be accessed 24/7 by staff and students. The system was programmed to interact dynamically with the user and to provide answers to context-specific questions. The web site layout is based on an accordion style menu. The VA is in the form of an avatar that interacts with the user in both text and speech.

Fig. 1.
figure 1

Virtual Assistant Interface

Full size image

To achieve the above a combination of applications were used:

  • Character Hosting from Media Semantics

  • Program-O

  • HTML - Hyper Link Markup Language

  • AIML - Artificial Intelligence Markup language

  • XAMPP to build the system on a localhost

Character Hosting was used to create and customise the avatar, its voice and accent, and add animations. Program-O has its own AIML interpreter, which was used to interpret the rules for user interaction with the avatar. It is coded in PHP and has an inbuilt relational database that can be used to store the interaction between the user and avatar. These can then be retrieved and analysed by the programmer in order to improve the responses and also provide the user with a log of their conversation. AIML code was used to enter the rules for topic-specific questions from the user and provide the knowledge base for the responses.

As mentioned earlier, the university agency targeted was the employability service, and we worked closely with a key member of staff within the service to learn about the nature of the advisory role, how a virtual assistant might augment that role and thereby to develop the content for the AIML rules.

5 Evaluation

To evaluate the pilot system we once again turned to a questionnaire, which was completed by 10 final year students after having being given the opportunity to use the system. The rationale for using final year students for the prototype evaluation was that the system was implemented within the limited domain of student employability, which was of particular interest to those in their final year. However, on reflection, a larger sample of students would have been preferable and consequently our results can only be viewed as indicative at this stage. The questionnaire comprised mainly closed questions, but students were also invited to add additional comments. Not all students answered all of the questions.

The survey revealed that half of the students found the avatar system either extremely easy or moderately easy to use; none of them found it difficult to use.

Eight out of nine students found the avatar’s responses to their questions very relevant, quite relevant or moderately relevant. This was encouraging, as the programming of the rules to produce a meaningful dialogue with the avatar was one of the most difficult aspects of the project. Despite the use of wildcards and pattern matching, and the limited nature of the domain, it was still difficult to anticipate and program meaningful reactions to all possible user queries.

Seven out of nine students found the avatar’s tone of voice either very appealing, quite appealing or moderately appealing. This was unexpected, as we were not entirely satisfied with the quality of the avatar’s voice and previous research [11, 12, 13] has found that subjects respond to artificial voices in the same way as they do with real human voices.

The positive reaction of the students to the avatar system was exemplified by their responses to a question about the extent to which avatar-based interfaces should be used for interacting with the university’s online services. Most of the students said that such systems should be used for all services, the majority of services or a lot of services, while one student thought they should be used sparingly. No students believed that avatars should never be used.

Students also gave us some useful feedback on the general layout of the interface and additional employability-related subject content that they would like to see included in the system. Some of their comments (in italics) on the nature of their interaction with the avatar were:

“Though it had limited responses, it gave useful information.”

“I like that the avatar reads the text to you, this makes the user fully understand and not miss certain words nor paragraphs. I also tried making a couple spelling mistakes such as adding extra letters to words in the question input tab, the system still provided rough the answer and/or rough guidance.” (sic)

“The system was relatively consistent and user friendly.”

These were encouraging, as they indicated that students could have a meaningful and valuable interaction with the system. Some responses were contradictory, for example: “…quick response to question…” but also “I would like the avatar response to be quicker than it is now.”

“… make the avatar more appealing, it looks bit dull at the moment” Although only one student made this point, it is generally accepted that it is challenging to make avatars express facial expressions appropriate to the words being spoken, which would be the best way to make them more appealing [16, 18].

“The avatar system is extremely user friendly and this is the very first time I am using it. Receiving a spoken response gives you a feeling that someone is physically there to help you.” This last response was very interesting; our intention with this system was to try to simulate the supportive dialogue that a student might have with a member of staff, and this comment suggests that we might, at least in part, have gone some way towards achieving this. A member of the employability service also evaluated the working prototype using a ‘think aloud’ protocol, and was satisfied that the system was fit for purpose.

6 Conclusions

We established that the introduction of a virtual assistant to supplement face-to-face advice was considered to be desirable by a significant sample of undergraduate students at every level and also by members of the university’s employability service.

We then successfully developed, in collaboration with the employability service, a virtual assistant using a rule-based engine and a text-to-voice avatar front end, to advise students on such matters as CV writing. Feedback from students’ evaluation of the system was generally positive, with some indication that our goal of providing an (albeit limited) out-of-hours version of the support provided by university employability service staff had been achieved. Feedback from the employability service on the prototype was positive and they foresaw using the VA in providing 24/7 student support for their service.

Our intention now is to roll out this technology to augment other university support services and also some aspects of the role of academic tutors. Ultimately, the success of the project will depend heavily upon the quality of the content and the sophistication of the interaction rules. Obviously, such systems are unlikely to pass the Turing test, but it is important that they are able to provide responses relevant to the user’s questions, to maintain a reasonably convincing dialogue and to guide the student towards the required information, and this pilot study has shown that, with careful design, this is possible.