Keywords

1 Introduction

Teamwork assessment as one of the competences more demanded by the labor market has become also a key competence to be developed in the educational institutions [1]. The acquisition of such competences requires students to work together, in groups as they will do possibly in their future work. Working in groups involves different type of activities, as can be work distribution, planning, development, review and publication of the results, etc. All of them have something in common, they require team members interaction and collaboration [2, 3].

If the educational institutions aim to facilitate the development of teamwork competence it is necessary to assess how students acquire it. In order to do so, several research works have shown that this cannot be evaluated only by considering each team results, but their interaction as a key element in Team Work Competence (TWC) development [4].

The interaction of the team members in face-to-face environments will probably carried out by talking. However, in an educational context collaboration is not something that will take place only in this way because: 1) discussions with peers are not always possible in classes specially with big groups; 2) in many cases collaboration goes beyond the educational institution where students are not together; and 3) students use other type of tools to interact between them, something that has been specially highlighted by the pandemic situation.

Taking this heterogeneity into account it seems clear that the evaluation of students’ interactions during the teamwork activities is something hard specially with big groups, so, it is necessary to use tools that facilitate such work. In [5] Fidalgo et al. the researchers describe a Learning Analytics (LA) tool that allows analyzing students’ interactions in forums. This was later tested successfully in other contexts [4,5,6,7,8,9,10]. However, a common complaint of the students involved was that they do not use commonly the forums to interact, that they employ other tools such as instant messaging tools. This leads to the implementation of two LA tools to evaluate instant messaging tools, one for analyzing students’ WhatsApp interactions and the other one for Telegram [11, 12]. They were successfully applied in two experiences and with this paper, we would like to compare both systems and to discuss which could be better for the evaluation of the individual acquisition of TWC.

In order to do so, the paper is structured as follows. Next section describes the research context describing and comparing WhatsApp and Telegram. Section 3 will explore and compare the LA tools developed for the analysis. Section 4 will discuss the implementations, comments on some preliminary evaluation results, and presents the conclusions.

2 Research Context

This section describes the concept of Instant Messaging tools (IM) and specially WhatsApp and Telegram apps.

2.1 Instant Messaging Tools

Nowadays one of the most common ways of communication is IM tools. These are online tools that facilitate synchronous communication among users that has become very popular with their inclusion in smartphones and the popularization of this kind of devices [13]. In fact, they are one of the most significant tools in our daily life and especially relevant for young people. They even prefer using IM apps than other communication tools such as phone calls or email [14, 15].

The popularity of these tools can be shown by the number of users they have. Considering stadista.com last report (October of 2020 - data based on We are Social, Kepios and other sources [16]), the most popular apps for IM were: WhatsApp with 2 billion users, Facebook messenger with 1.3 billion, WeChat with around 1.2 billion, QQ mobile with 648 million, Snapchat with 433 and Telegram with 400.

In education, it is the most popular communication among students. They consider IM tools as very straightforward apps [17] with which they can chat individually or in groups, share images, videos, voice, talk, etc. [13]. In educational contexts, IM tools have been applied for different purposes although the most common aim is to improve communication experience among students and teachers [18,19,20,21,22,23]. These experiments have shown that IM facilitates knowledge sharing and enhances interactivity between peers; increases the sense of being present at a place; support collaboration and makes possible to interact anywhere and anytime [23, 24]. There are nonetheless tradeoffs, the use of IM tools can distract students from learning because it can be used for educational purposes, but it is not something easy to control [25, 26]. So, if we want to apply these tools in education, which can be really interesting given the above describe advantages, it is necessary to control how students are using them.

There are different possible ways to apply IM in education but the most common are using them as a communication channel for students’ groups. Such interaction can be later analyzed from a quantitative perspective [24] or taking into account the contents of the messages [27]. In the case of this work, the idea is to use it for assessing teamwork which has been seen as a possibility in other of the authors works, but that is not very common.

2.2 WhatsApp vs Telegram

Given this context, we can understand that can be possible and positive to employ IM tools in education. Now it is time to decide which tool to use. The present authors have carried out experiments and adaptations both for WhatsApp and for Telegram, so now we compare both technologies and later how they can be adapted for the educational contexts.

WhatsApp

It was created as a free instant messaging tool that uses Internet to allow users to send texting messages without character limits and share with other images, audios, videos, weblinks, documents, etc. [28]. WhatsApp also provides additional social information to its users, e.g., contacts can see when their friends are online, when they are typing and when they last accessed the application. Finally, WhatsApp provides delivery notifications, highlighting when a message is sent and when it is delivered to the recipient’s device [29]. Recently it was acquired by Facebook.

As we have seen before it is the most popular IM app but, why is people adopting it? Among other reasons because: it is free; the possibility to send unlimited messages; the sense of synchronous communication; it is trendy; to interact with a group of persons and because of the sense of privacy compared with other social networks [29].

Regarding the application of WhatsApp in education, there are several studies describing the possibilities it provides in this field [30,31,32], samples about how to use it for students and teachers communication [20, 24], and others related to the development of specific competences [33, 34] which also could include the works of the authors about teamwork interaction assessment.

Telegram

Telegram is another very popular and free IM service based on an open-source platform. Services are similar to those provided by WhatsApp, although there are differences that we will comment later. Telegram was developed in August 2013 by russian-born entrepreneur Pavel Durov [35]. Telegrams users have grown in the last years, passing from 200 million users in 2018 to 400 in 2020 [36].

The use of Telegram includes several specific services such as the use of Telegram ID so someone can contact the user without a phone number, the use of bots, the presence of an open API to integrate the tool and messages with other systems, the access through a great number of platforms as it is stored in the cloud, etc. [35].

Regarding the possible uses of Telegram in Education it, as WhatsApp and other IM tools is mainly used as communication channel to support educational activities in several fields [37,38,39,40], although in this case, the possibility to use channels or bots [41] changes the typical way in which IM tools used to be employed.

Differences Among Telegram and WhatsApp

It is necessary to point out that these two IM platforms share a lot of functionalities but also present important differences. Although, as they are continuously evolving, the functionalities included in each of them, tend to include what their users demands, which means that what now is not present in one of the platforms could be included in the near future. For instance, WhatsApp does not include stickers or animated gifs but they have been recently implemented or Telegram that does not include video calls but have included them in August 2020.

The following issues are based on several comparisons, we are only focused on differences defined by tech web pages [42,43,44], similarities are ignored:

Privacy and Security. Although both IM tools end-to-end encryption, so conversations can only be available for the receiver and sender, Telegram adds a functionality that are secret chats where screenshots are not possible and messages are self-destructed after a time.

Compatible platforms. Telegram is defined as a cloud-based system, so it is easier to access it from any platform and from different devices at the same time. WhatsApp also includes this functionality as WhatApp web, but it can be only used in a device at the same time and it is linked to a phone number.

  • Telegram ID. Telegram provides to the users an ID, so the user account does not need to be linked to a phone number (it is only necessary on the installation) and it is possible to share only the TelegramID and not the phone number with other contacts.

  • Groups and Channels. WhatsApp facilitates groups for 256 users while Telegram allows in the last versions up to 200.000. In addition, Telegram allows the user to be subscribed or to create Channels. They are a tool for broadcasting public messages to large audiences, where the message is signed with the channel name and not the username.

  • API. WhatsApp is a proprietary tool so if you want to use its API it is necessary to pay for it, it is called WhatsApp Bussiness API [45]. On the contrary, Telegram is open source, so it provides an API [46] that facilitates access to different functionalities and the definition of tools that use it.

  • Bots. WhatsApp is developing currently bots, but they have this functionality available as part of the WhatsApp Bussiness API commented above. Telegram facilitates the definition of bots [47] and their integration in the groups and channels, there exist a wide number of bots with different functionalities.

  • Polls. Telegrams provide a functionality to include polls as messages in your chats. In Whatsapp it is necessary to define them externally and embed the result as a message.

  • Other. There are other functionalities different in both tools, as the type of emojis, the way in which backups are carried out, the possibility to pin or edit messages, etc.

It should be noted that, independently of the differences, nowadays WhatsApp is the most popular IM tool. Although Telegram has duplicated the number of users in one year as shown by statista reports [36, 48].

3 LA Solutions Based on Telegram and WhatsApp

In order to evaluate TWC individual acquisition we have employed CTMTC in several experiments but, as commented above, the students claim that their conversations using a forum were not natural. This leads to the definition of tools to facilitate this assessment when the interactions are carried out using WhatsApp or Telegram. In this section, we describe and compare both solutions. In any of them, the idea is to gather the messages and explore them with a LA tools in order to reduce the time required to evaluate the individual work in each group [5].

3.1 LA Solution for WhatsApp

This solution has been described in two previous works. It consists of the implementation of two different components, a web plugin included in a Learning Management Course – LMS (Moodle in this case) and a LA tool. The idea is that the teacher will instantiate a TW activity and the students can use the plugin to upload a text file with their WhatsApp conversation. The plugin parses the information and stores it in a new table on Moodle database. The information stored can be later accessed by a LA tool using Moodle web services layer. The teacher could check what the students have done in the LA and evaluate them according to a rubric. The data shown by the LA will be the same for Telegram and for WhatsApp. Figure 1 shows the component distribution for this option.

Fig. 1.
figure 1

Description of the components involved in the LA tool implementation for WhatsApp (obtained from [11]).

Full size image

3.2 LA Solution for Telegram

In this case the solution is based on Telegram bots technology, and it is independent of any LMS. In order to develop it a bot was created that uses webhook methods to gather the messages and facilitate a real-time analysis of the activity that take place in the conversations where the bot is included. So, the teacher only needs to provide the students with some guides to use the bot, they include them in their group and the messages can be gathered and later analyzed in the LA. In this case the LA includes the same date considered in the WhatsApp experience but there will be no integration with the LMS. The deployment diagram of this solution is shown in Fig. 2.

Fig. 2.
figure 2

Deployment diagram with the different components (obtained from [12]).

Full size image

3.3 Differences Between Both Solution

Technically, both solutions are ok and in both cases the LA is an external component that should present the data to facilitate the teacher making decisions. Regarding the differences we can find several:

  • Isolated component or integration with the LMS. In the WhatsApp solution the component is integrated into Moodle. This means that the teacher should instantiate the activity and the students should access to it to parse their conversations. In addition, this implies that the information parsed is stored in the institutional LMS (commented below). In the case of Telegram there is not a component in this sense beyond the bot. The integration of the component into Moodle presents several advantages, as the centralization of all the activities and data, but also means that it is necessary that teachers and students interact with it. In the case of Telegram, it is more straightforward. The teachers need only to explain how the bot works and the students to include it into the group and execute some commands.

  • Data stored in the LMS or on an external Server. This can be interesting because the information will be stored in the LMS and transferred via web services to the LA tool, so the institution could “control” the data. The problem is that this data could be attacked when it is transferred. In the case of Telegram this attack it is also possible, but the information is going to be stored in an external server that we can control. In this case WhatsApp drawback could be that the educational institution does not allow to access to the LMS database.

  • Parse the information or not. In the case of WhatsApp, we need to parse the data, because the text conversation includes phone numbers and in the case of teachers it is not necessary nor desirable this information but a student id in order to know the author of each message. In the case of Telegram, as it uses TelegramIDs, the phone number is not included in the data and the student only needs to execute a command in the group to associate their PersonalID with the TelegramID. This means less work to be done by the student and more privacity as the phone number is not present in any part of the evaluation process.

4 Discussion and Conclusions

This work has explored two very popular IM Tools that can be used by students and teachers to interact in teamwork tasks. The evaluation of the TWC requires to explore such interaction so, in order to do this, we developed two different LA tools. The paper has analyzed both the technical differences of the IM tools and the differences between the LA Tools. Is this enough to answer the question about which is the best tool? We have tested both tools in two experiments carried out with students from a second course subject of Computer Science degree. We are still analyzing the results of the second experiment, but with more than 100 students involved per each of them, we can say that the question cannot be answered flatly, and it depends on several issues:

  1. 1.

    What tool or tools are the students using to interact? Using WhatsApp or Telegram can be seen as a trendy solution, but they are not the only possible communication ways of the students. If they employ other tools, then the interaction cannot always be tracked or analyzed. As we are applying this in a controlled educational context, we have allowed the students to use an asynchronous tool for communication (Moodle forums) or a synchronous (WhatsApp). Most of the students involved decide to use the synchronous tool (more than a 90%) although, especially those with less interaction, comment that they employ other tools to chat with their team partners, such as Skype, Discord, Facebook, etc.

  2. 2.

    If the students were used to employ WhatsApp or Telegram they were happy using them, but if they have not used these tools before they say that this cause them more effort.

  3. 3.

    Students suggested the analysis of audio as a possible research work and although WhatsApp and Telegram allow audio and videocalls, they were not defined as voice channels. Anyway, this functionality should be taken into account in future developments

  4. 4.

    The average number of messages per student has been increased in comparison with forums with an average of 216.3 messages per student in WhatsApp vs 164.3 in Telegram.

  5. 5.

    Some of the conversations seem artificial and only to answer to the assignment requirements, that is, the students were not using these tools for a real interaction.

  6. 6.

    Teachers consider that any of the tools can help the to assess students’ interaction, however, it would be useful to include emojis analysis and the application of natural language processing techniques over the messages, so it was possible to take into account other competences related to TWC or with leadership [49, 50].

  7. 7.

    From the programmers’ point of view they prefer a Telegram solution because in this way there is not constraints related with the educational institution databases. Although this requires securing properly the data in the server where it is going to be stored.

Given these issues, it is clear that it is necessary to continue working in this line, taking into account the different suggestions and promoting among the students that they should interact using the IM as they are interacting in real conversations, which use to be one of the hardest parts of the work.

Answering the question raised by this paper is not easy, the use of synchronous tools is very interesting in the context of teamwork, but in anyway the students should be motivated. If we use WhatsApp or Telegram probably is not they key issue, but how to enhance and promote real students’ interactions.