Abstract
In our present professional and educational contexts one of the key competences to assess is the teamwork competence. However, this require not only to explore the development of the competence by a group but how it is acquired by each of its members. In order to do so it is necessary to analyze several issues and one of the most relevant is students’ interaction. The problem is that the tools that students employ to interact in their learning context are not those that they use in their daily life. That’s why it is necessary to explore the students’ interaction that happens through instant messaging tools, and the most popular is WhatsApp. This paper explores the problem of analyzing WhatsApp messages and how to compile these evidences in such a way that they can be explored through a learning analytics tool. The implementation shows that the use of WhatsApp messages is possible but requires to take into account other issues such as how to manage interoperability and how to deal with user sensitive information.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Our current society, that may be considered as the Information society, the Knowledge society or the Technological society; requires better prepared professionals in order to address the problems and requirements of a continuously changing world. These professionals should acquire several competences while they are learning in order to succeed in a very competitive environment. One of the key competences to acquire is teamwork competence (TWC) which is promoted by the educational institutions and very demanded by industry [1]. The acquisition of this competence is especially relevant because: (1) TWC development contribute to improve students’ learning as they need to share information and discuss among them, which facilitates building mental models in a cooperative way [2, 3]; (2) in institutions and organizations is common to work in teams, so it is desirable that all the professionals know how to develop their work in this way [4]; (3) Bologna process understands TWC as one of the key competences to acquire by the students in Higher Education [5].
Although institutions are devoting many effort and investing great quantities of resources to promote and foster TWC, it is necessary to assess if the competence has been acquired or not. This is a difficult task that requires of the evaluation of multiple evidences. Most of the time the assessment of TWC is determined by the final result or work that teams deliver, ignoring what happens during the different stages of the process or variability between the amount and relevance of individual contributions [6].
It is necessary to apply a methodology that allows not only assessing the final result of a group but the individual acquisition of teamwork assessment. In this sense there are three possible categories of instruments that can be used [7]:
-
Observational rating scales. These are standardized measurement protocols, which train observers to record and rate observable team behavior. Some examples could be Behaviorally Anchored Rating Scales BARS [8] or Communication and Teamwork Skills (CATS) [9]. The problem of this type of methodology is that is very hard to apply them when the number of groups to evaluate is high and because is difficult to assess part of the teamwork, especially when it is not carried out in a controlled environment.
-
Team self-assessment. These methodologies could be also applied to try to assess teamwork. With them the students rate their own work in the group and also their peers work. Examples of this kind of instruments could be the Hackbert questionnaire for peer and self-assessment [10], the instrument defined by Ríos-Carmenado et al. [11] or the Team Work Behaviour Questionnaire (TWBQ) [12]. In this case the problem is that peer evaluation can be biased by the context of the student and the relationship with her peers.
-
Event-based approach. These techniques were defined to deal with performance evaluation in complex simulation scenarios [13, 14]. In this approach critical events which provide opportunities to perform key teamwork competencies are in the simulated scenarios. The problem of this methodology is that it cannot easily be applied all kind of educational approaches.
Most of these methodologies present different problems to evaluate the individual acquisition of TWC. Given this situation the authors of this work decide to apply Comprehensive Training Model of the Teamwork Competence (CTMTC) [15, 16] in several academic contexts. This is a methodology that analyses the learning evidences recorded by the IT-based learning tools that the teams use when developing a project or a learning activity [16]. CTMTC is based on a set of stages adapted from the International Project Management Association (IPMA) [17]. The students are going to complete these stages in order to develop their project. CTMTC is a methodology focused on the evaluation of issues of group-based learning: teamwork phases (mission and goals, responsibility maps, planning, implementation and organization of documentation), collaborative creation of knowledge, and cloud computing technologies (wikis, forums, social networks and cloud storage systems) [16]. The people in charge of the application of this methodology can track continuously what students are doing and how they are collaborating to develop the project, and can analyze how an individual is contributing to the group work. In addition it facilitate teachers to do partial summative assessments of TWC [18]. The method has been tested in very different contexts with high rate of success and acceptation [6, 18,19,20,21,22,23], and it was possible to adapt it easily which show how flexible it is.
But not all that happens in those experiments was good. One of the main problems found with these experiences were the tools employed to show the communication between team members. The most common tool to use was the forum provided by a Learning Management System, more specifically Moodle forum. Moodle forum is a very powerful tool that can be used to carry out several type of learning activities [24]. However, students were used to employ synchronous tools more than the asynchronous ones, since their messages in in forums look artificial, it does not reflect the real interaction that they could have in other applications such could be an instant messaging client as is WhatsApp. This was suggested in the qualitative assessment carried out in the above cited experiments and fits with other research conclusions [25].
Given this context, the present work aims to make possible importing the WhatsApp messages of the members of a team that is applying CTMTC methodology. This is a complex task because of three main reasons: (1) WhatsApp is a commercial tool that is not included in the Learning Management System but in the personal devices of each student; (2) The information recorded in WhatsApp links a person with a telephone number which is sensitive information that should not be stored; and (3) it is necessary to study if it is possible to apply the same learning analytics techniques and tools with the data source obtained from WhatsApp than from the information recorded into the LMS.
This paper is going to deal with these three issues. It is structured as follows: Sect. 2 analyses other existing works that employs instant massaging such as WhatsApp in learning activities; Sect. 3 describes how facilitate the connection between the LMS and WhatsApp; and Sect. 4 how the tool is implemented and the information is parsed. Finally, some conclusions are posed.
2 Research Context
Nowadays one of the technological tools most used are Instant Messaging (IM) apps. These are online tools that facilitate a synchronous communication among users that has become very popular with their inclusion in the smartphones and the popularization of this kind of devices. In the educational field, IM apps allow students to exchange text, chat in group, share images, videos, voice and even talk.
In 2018 the most popular apps for IM were: WhatsApp with 1,5 billion users, Facebook messenger with 1,3 billions, WeChat with around 1 billion, QQ mobile with 803 millions, Skype with 300, Snapchat with 291, Viber with 260, Line with 203 and Telegram with 200 (stats provided by stadista.com based on We are Social, Kepios and other sources [26]).
These stats show how significant is IM in individuals daily live something especially relevant when focusing in students, that prefer to use these apps instead of email or phone calls to stay in touch with their friends [27, 28]. Students do not understand IM as technology, a term associated in many people’s minds with objects that are complicated and difficult to understand or operate, but as common communication way [29].
IM apps have been applied in learning contexts in different ways, but one of the most common is helping to improve students communication with peers and faculty [30,31,32,33,34,35]. From this experiments it is possible to say that IM facilitates interactivity between peers and knowledge sharing, increases participants sense of presence, fosters collaboration and makes possible ubiquity [35, 36]. However, if the learners use these tools when studying, and not necessarily with educational proposes, they may have a negative effect, because IM apps can be a distraction in the learning process [37, 38]. This means that IM apps could be interesting tools in educational activities but they should be used in a controlled way and with educational proposes.
One of the main potential contributions of IM apps is that it can enhance communication between students which is in one of the key issues for the acquisition of TWC. In this sense, there are several cases where instant messaging is applied as a communication channel for students groups [36] and in some the contents of the interaction is analyzed [39]. However, in those works the learning evidences are not used to assess TWC acquisition as the author of this paper aims to do.
Given this context it is clear that IM can be used to communicate groups of students, but, What IM app should be used? In this sense the present work explores the use of WhatsApp as one the most popular IM tools. There are several studies about the possibilities that WhatsApp provides in educational contexts [40,41,42], samples of the application of WhatsApp for communication [32, 36] and other related with the development of specific competences [43, 44].
This work goes beyond the existing research on this field, because it aims to employ WhatsApp to facilitate cooperation among the members of students’ groups, that work together to develop a project, and later allow analyzing the interaction of the team members in order to assess the individual acquisition of the TWC.
3 The Interconnection Between WhatsApp and the LMS
In order to design the possible implementation of the connection between the learning analytics tool and WhatsApp it is necessary to consider the main components of this interaction:
-
Learning Analytics Tool (LA Tool). It is an ad-hoc defined tool that gathers, analyze and represent information about the students’ interaction in groups when they are applying CTMTC to address their projects. The system allows choosing a forum within the course and then display the data for the student–student interactions, with three different view modes [19]:
-
Forum view. With general stats about the interaction in the forums by each group and by students.
-
Team view. That provides information for a specific team in all the threads of a forum
-
Thread view. Which shows specific information for a group and a thread
-
The information is gathered by using an ad-hoc defined Moodle webservice and a client. The client will request the information to Moodle by using the webservice layer, compile this information and preprocess it. Figure 1 shows the information represented by the client for a specific team.
-
Moodle. It is one of the most popular LMS and is used in lot of institutions all around the world [45,46,47]. Moodle, as other LMS, provides tools to support the teaching and learning processes. Between these tools one of the most common are forums. In CTMTC the forum is used as a way to gather the interactions that the team members have among them. This information is accessed by the LA tool. For this project the idea is to store WhatsApp evidences also into Moodle database so they can be easily accessed by the LA client without the necessity of dealing with a different database.
-
WhatsApp. It is the tool where the interaction of students could happen. It is necessary to find a way to send the interactions to Moodle database. There are three possible options:
-
Modify WhatsApp to include interoperability specifications. This is not possible because it is a proprietary software tool, so modifications are not allowed.
-
Define a middleware that gathers information from WhatsApp and implements interoperability specifications as the proposed by Conde et al. in other works [48, 49]. The problem of this solution is that there is not an official WhatsApp Open API and only WhatsApp Business API that can be employed to define commercial solutions. In this case the mediator could not be the best option because it is necessary to pay to access to the API.
-
Define a component into Moodle that gathers the WhatsApp evidences and store them in Moodle database, so later could be represented in the LA tool client.
-
Given these options the best and more affordable solution is the last one. The idea is to define a widget with a form that can be installed in Moodle. In such component students should upload a text file with the WhatsApp conversation (a WhatsApp default functionality) and the information will be processed and stored into Moodle.
4 The Implementation and Information Process
Once made the decision to define a Moodle widget, authors should decide what type of widget develop. They considered that the best option could be a module, that would be instanced in the course. In this module the teacher should decide the forum to which the conversations will be associated. Through the module, students could upload a text file with the conversation and the information will be stored in the database. However, an important issue was detected related with WhatsApp messages structure. Figure 2 shows the structure of a WhatsApp message. The first part of the message is the date, after a “-” it is possible to see the phone number of the person that is speaking or her contact name.
This means that the WhatsApp conversation does not include students’ IDs, which could be a problem to link the messages with the specific students in the LMS database. In addition, storing sensitive information, such as a phone-numbers with students’ IDs, can break the EU General Data Protection Regulation (GDPR). Given this situation the text file should be parsed. The students are going to associate, when uploading the file, the ID to a specific phone number or contact name. With this information the parser will extract messages and associate them to the students’ IDs. Figure 3 shows an example of the form included in the module.
Figure 4 shows a Business Process Model Notation (BPMN) diagram that shows the process of uploading and storing the information. In such diagram it is possible to see that the student export and upload a file and fulfill a form to link user ids with contact names or phone numbers. If the form is properly fulfilled the information is parsed and stored in the data base. If there were errors these will be shown to the user that must fulfill again the form.
The last part of the development consists of the integration of new methods into Moodle Web Services that allow retrieving WhatsApp messages for a specific group and to modify the LA tool client to facilitate data representation. Authors are currently working on this part.
5 Conclusions
TWC acquisition by an individual is closely related to her interaction with her peers. This kind of interaction could be carried out in forum posts, blogs or with other tools linked to an educational environment. However, students do not interact usually by using these tools. It is much more common that they employ IM tools and the most popular tool nowadays is WhatsApp.
In this paper we have explored how WhatsApp is employed in learning activities and we have provided a way to store WhatsApp conversations in the LMS database so later it can be explored through a LA tool in order to evaluate real students’ interactions. This required to define a tool to upload conversations, parse sensitive information and store it in a coherent way with Moodle database structure.
From the experience it has been possible to say that the interoperability between tools that belong to the student informal learning context can be combined with those from the formal one, but it is necessary an intervention by the student in order this can be done in the proper way. This fits with some previous works [48, 50].
The next step is to validate the integration of the tool through some experiments where we will compare the results about individual TWC acquisition distinguishing between groups that upload WhatsApp conversation and those which prefer using the forum. Moreover, students that employ WhatsApp will be surveyed in order to know their perception about the integration carried out and what other tools they would use.
References
Colomo-Palacios, R., Casado-Lumbreras, C., Soto-Acosta, P., García-Peñalvo, F.J., Tovar-Caro, E.: Competence gaps in software personnel: a multi-organizational study. Comput. Hum. Behav. 29, 456–461 (2013)
Leidner, D.E., Jarvenpaa, S.L.: The use of information technology to enhance management school education: a theoretical view. MIS quarterly 19, 265–291 (1995)
Vogel, D.R., Davison, R.M., Shroff, R.H.: Sociocultural learning: a perspective on GSS-enabled global education. Commun. Assoc. Inf. Syst. 7, 1–41 (2001)
Iglesias-Pradas, S., Ruiz-de-Azcárate, C., Agudo-Peregrina, Á.F.: Assessing the suitability of student interactions from moodle data logs as predictors of cross-curricular competencies. Comput. Hum. Behav. 47, 81–89 (2015)
European Ministers Responsible for Higher Education: The Bologna Process 2020 - The European Higher Education Area in the new decade. Communiqué of the Conference of European Ministers Responsible for Higher Education, Leuven and Louvain-la-Neuve, 28–29 April 2009. https://goo.gl/ABJX7x
Conde, M.Á., Rodríguez-Sedano, F.J., Sánchez-González, L., Fernández-Llamas, C., Rodríguez-Lera, F.J., Matellán-Olivera, V.: Evaluation of teamwork competence acquisition by using CTMTC methodology and learning analytics techniques. In: Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality, pp. 787–794. ACM, Salamanca (2016)
Rosen, M.A., et al.: Tools for evaluating team performance in simulation-based training. J. Emerg. Trauma Shock 3, 353–359 (2010)
Schwab, D., Heneman III, H.G., DeCotiis, T.A.: Behaviorally anchored rating scales: a review of the literature. Person. Psychol. 28(4), 549–562 (2006)
Frankel, A., Gardner, R., Maynard, L., Kelly, A.: Using the communication and teamwork skills (CATS) assessment to measure health care team performance. Jt. Comm. J. Qual. Patient Saf. 33, 549–558 (2007)
Hackbert, P.H.: Building entrepreneurial teamwork competencies in collaborative learning via peer assessments. J. Coll. Teach. Learn. 1, 39–52 (2004)
Ríos-Carmenado, I., Figueroa-Rodríguez, B., Gómez-Gajardo, F.: Methodological proposal for teamwork evaluation in the field of project management training. Procedia Soc. Behav. Sci. 46, 1664–1672 (2012)
Tasa, K., Taggar, S., Seijts, G.H.: The development of collective efficacy in teams: a multilevel and longitudinal perspective. J. Appl. Psychol. 92, 17–27 (2007)
Dwyer, D.J., Oser, R.L., Salas, E.: Event-based approach to training (EBAT). Int. J. Aviat. Psychol. 8, 209–221 (1998)
Lane, N.E., Salas, E., Franz, T., Oser, R.: Improving the measurement of team performance: the TARGETs methodology. Mil. Psychol. 6, 47–61 (1994)
Fidalgo-Blanco, Á., Lerís, D., Sein-Echaluce, M.L.: Monitoring indicators for CTMTC: comprehensive training model of the teamwork competence in engineering domain. Int. J. Eng. Educ. (IJEE) 31, 829–838 (2015)
Lerís, D., Fidalgo, Á., Sein-Echaluce, M.L.: A comprehensive training model of the teamwork competence. Int. J. Learn. Intellect. Cap. 11, 1–19 (2014)
NCB.- Bases para la competencia en dirección de proyectos. http://www.lpzconsulting.com/images/CP-_Trabajo_en_Equipo.pdf. Accessed 28 Feb 2014
Séin-Echaluce, M.L., Fidalgo Blanco, Á., García-Peñalvo, F.J., Conde, M.Á.: A knowledge management system to classify social educational resources within a subject using teamwork techniques. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2015. LNCS, vol. 9192, pp. 510–519. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20609-7_48
Fidalgo-Blanco, Á., Sein-Echaluce, M.L., García-Peñalvo, F.J., Conde, M.Á.: Using learning analytics to improve teamwork assessment. Comput. Hum. Behav. 47, 149–156 (2015)
Fidalgo, A., Leris, D., Sein-Echaluce, M.L., García-Peñalvo, F.J.: Indicadores para el seguimiento de evaluación de la competencia de trabajo en equipo a través del método CTMT. Congreso Internacional sobre Aprendizaje Innovación y Competitividad - CINAIC 2013, Madrid (2013)
Sein-Echaluce, M.L., Fidalgo-Blanco, Á., García-Peñalvo, F.J.: Students’ knowledge sharing to improve learning in engineering academic courses. Int. J. Eng. Educ. 32, 1024–1035 (2016)
Conde, M.A., Colomo-Palacios, R., García-Peñalvo, F.J., Larrucea, X.: Teamwork assessment in the educational web of data: a learning analytics approach towards ISO 10018. Telematics Inform. 35(3), 551–563 (2018)
Conde, Miguel Á., Hernández-García, Á., García-Peñalvo, F.J., Fidalgo-Blanco, Á., Sein-Echaluce, M.: Evaluation of the CTMTC methodology for assessment of teamwork competence development and acquisition in higher education. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2016. LNCS, vol. 9753, pp. 201–212. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39483-1_19
González, I., Martín, R.M., García, F.J., Seoane, A.M., Conde, M.Á.: Interacción, aprendicaje y enseñanza basada en foros. Un caso de estudio sobre la plataforma ClayNet. In: Redondo-Duque, M.Á., Bravo-Santos, C., Ortega-Cantero, M. (eds.) Diseño de la Interacción Persona-Ordenador: Tendencias y Desafíos. Actas del VII Congreso Internacional de Interacción Persona-Ordenador, Interacción 2006, pp. 303–313, Puertollano, Ciudad Real, Spain (2006)
Alghamdi, E.A., Rajab, H.: Unmonitored students self-created WhatsApp groups in distance learning environments: a collaborative learning tool or cheating technique. Int. J. Eng. Educ. 5, 71–82 (2016)
Most popular global mobile messenger apps as of October 2018 Statista (2019). https://www.statista.com/statistics/258749/most-popular-global-mobile-messenger-apps/. Accessed 5 Feb 2019
Carnevale, D.: Email is for old people. Chron. High. Educ. 53, A57 (2006)
Junco, R., Mastrodicasa, J.: Connecting to the net.generation: what higher education professionals need to know about today’s students. NASPA, National Association of Student Personnel Administrators, Student Affairs Administrators in Higher Education, US (2007)
Lewis, C., Fabos, B.: Instant messaging, literacies, and social identities. Read. Res. Q. 40, 470–501 (2005)
Hrastinski, S., Edman, A., Andersson, F., Kawnine, T., Soames, C.-A.: Informal math coaching by instant messaging: two case studies of how university students coach K-12 students. Interact. Learn. Environ. 22, 84–96 (2014)
Cifuentes, O.E., Lents, N.H.: Increasing student-teacher interactions at an urban commuter campus through instant messaging and online office hours. Electron. J. Sci. Educ. 14, 1–13 (2010)
Smit, I., Goede, R.: WhatsApp with BlackBerry; can messengers be MXit? A philosophical approach to evaluate social networking sites. Cape Peninsula University of Technology (2012). https://repository.nwu.ac.za/handle/10394/13628
Sweeny, S.M.: Writing for the instant messaging and text messaging generation: using new literacies to support writing instruction. J. Adolesc. Adult Lit. 54, 121–130 (2010)
Lauricella, S., Kay, R.: Exploring the use of text and instant messaging in higher education classrooms. Res. Learn. Tech. 21, 1 (2013)
Klein, A.Z., da Silva Freitas, C.J., da Silva, J.V.V.M.M., Barbosa, J.L.V., Baldasso, L.: The educational affordances of mobile instant messaging MIM: results of WhatsApp used in higher education. Int. J. Dist. Educ. Tech. 16, 51–64 (2018)
Bouhnik, D., Deshen, M.: WhatsApp goes to school: mobile instant messaging between teachers and students. J. Inf. Technol. Educ. Res. 13, 217–231 (2014)
Fox, A.B., Rosen, J., Crawford, M.: Distractions, distractions: does instant messaging affect college students’ performance on a concurrent reading comprehension task? Cyberpsychol. Behav. 12, 51–53 (2009)
Junco, R., Cotten, S.R.: Perceived academic effects of instant messaging use. Comput. Educ. 56, 370–378 (2011)
Gronseth, S., Hebert, W.: GroupMe: investigating use of mobile instant messaging in higher education courses. TechTrends 63, 15–22 (2019)
Willemse, J.J.: Undergraduate nurses reflections on WhatsApp use in improving primary health care education. Curationis 38, 1–7 (2015)
Barhoumi, C.: The effectiveness of WhatsApp mobile learning activities guided by activity theory on students’ knowledge management. Contem. Educ. Technol. 6, 221–238 (2015)
Aljaad, M., Hamad, N.J.E.: Whatsapp for Educational Purposes for Female Students at College of Education-King Saud University. Education 137, 344–366 (2017)
Awada, G.: Effect of WhatsApp on critique writing proficiency and perceptions toward learning AU - Awada. Ghada. Cogent Education 3, 1–25 (2016)
Andújar-Vaca, A., Cruz-Martínez, M.-S.: Mobile instant messaging: WhatsApp and its potential to develop oral skills. Comunicar 25, 43–52 (2017)
Cole, J., Foster, H.: Using Moodle. O’Really, Sebastopol (2007)
Al-Ajlan, A., Zedan, H.: Why Moodle. In: 12th IEEE International Workshop on Future Trends of Distributed Computing Systems, pp. 58–64. Kunming (2008)
Molist, M.: Moodle llena la geografía educativa española de campus virtuales. Diario el Pais (2008). http://www.elpais.com/articulo/portada/Moodle/llena/geografia/educativa/espanola/campus/virtuales/elpeputec/20081204elpcibpor_1/Tes
Conde, M.Á., García-Peñalvo, F.J., Rodríguez-Conde, M.J., Alier, M., García-Holgado, A.: Perceived openness of learning management systems by students and teachers in education and technology courses. Comput. Hum. Behav. 31, 517–526 (2014)
Conde, M.Á., García-Peñalvo, F.J., Alier, M., Mayol, E., Fernández-Llamas, C.: Implementation and design of a service-based framework to integrate personal and institutional learning environments. Sci. Comput. Program. 88, 41–53 (2014)
García-Peñalvo, F.J., et al.: TRAILER project (Tagging, recognition, acknowledgment of informal learning experiences) a methodology to make visible learners’ informal learning activities to the institutions. J. Univers. Comput. Sci. 19, 1661 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Conde, M.Á., Rodríguez-Sedano, F.J., Rodríguez-Lera, F.J., Gutiérrez-Fernández, A., Guerrero-Higueras, Á.M. (2019). Analyzing Students’ WhatsApp Messages to Evaluate the Individual Acquisition of Teamwork Competence. In: Zaphiris, P., Ioannou, A. (eds) Learning and Collaboration Technologies. Ubiquitous and Virtual Environments for Learning and Collaboration. HCII 2019. Lecture Notes in Computer Science(), vol 11591. Springer, Cham. https://doi.org/10.1007/978-3-030-21817-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-21817-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-21816-4
Online ISBN: 978-3-030-21817-1
eBook Packages: Computer ScienceComputer Science (R0)