Keywords

1 Introduction

Serious game analytics is an emerging research field, which aims at turning gameplay data into “valuable analytics or actionable intelligence for performance measurement, assessment, and improvement” [1]. Not only does this high-level goal require the need for developing serious games able to provide didactically relevant quantitative information, but also efficient integration of modules for analytics collection and management [2].

Dealing with big data is a general and common problem, particularly in the context of the Internet of Things (IoT), for which several solutions have been developed, especially using cloud-based frameworks. In this paper, we are interested in investigating the application to the serious game context of Measurify (previously known as Atmosphere), an open source IoT framework built around the concept of measurement, which looks quite relevant to the goal of assessing and supporting a learner [3].

Actually, that article already briefly reports the experience of a University spin-off company applying Measurify to design a data management model for the use case of a 3D virtual reality simulator for emergency room personnel instruction. The goal of the instructional tool was to assess the performance of a doctor by evaluating the effects of his interventions on the various patients. To this end, the Measurify data model was designed to record (i) the state of a patient, that is characterized by a set of time-evolving parameters; (ii) the actions performed by the doctor (e.g., how he interacted with the available simulation tools); and (iii) the events in the simulation (e.g., changes in medical equipment availability).

In this paper, we intend to go more in depth about the utilization of Measurify as an analytics engine, particularly investigating its extension towards edge computing [4], namely the recently released Edgine module [5], aimed at supporting a configurable provision of measurements from the field.

The remainder of the paper is organized as follows. Section 2 provides the related work, while Sect. 3 and 4 describe the Measurify and Edgine systems, respectively. Section 5 is devoted to the application of the IoT framework to a gaming environment. Section 6 draws the conclusions on the presented work and outlines possible directions for future research.

2 Related Work

Serious games analytics is a research field involving empirical research methodologies, including existing, experimental, and emerging conceptual frameworks, from various areas, such as: computer science, software engineering, educational data mining and statistics information visualization [1].

The literature provides several examples of serious game analytics (or game learning analytics) applications. [6] explored players’ gameplay patterns to understand player dropout in the Quantum Spectre science game. Based on their results, the authors argue that modeling player behavior can be useful for both assessing learning and for designing complex problem solving content for learning environments. [7] present a novel method that suggests curricular sequencing based on the prediction relationship between math objectives. The authors argue that their method can potentially be applied to data from a wide range of games and digital learning platforms, enabling developers to better understand how to sequence educational content. [8] explore existing log files of the VIBOA environmental policy game. Our aim is to identify relevant player behaviours and performance patterns. The correlation analysis suggests to the authors a behavioural trade that reflects the rate of “switching” between different game objects or activities. The authors also established a model that uses switching indicators as predictors for the efficiency of learning. [9] developed a mobile game to support the transfer of theoretical knowledge on resuscitation training in case of cardiac arrest. To analyse a large and heterogeneous (in terms of sources and quality) data-set collected from 171 players, the authors applied different types of data modeling and analyses. This approach showed its usefulness and revealed some interesting findings.

[10] discuss a conceptual model (ecosystem and architecture) aimed to highlight the key considerations that may advance the current state of learning analytics, adaptive learning and SGs, by leveraging SGs as an suitable medium for gathering data and performing adaptations. [11] describe two key steps towards the systematization of game learning analytics: 1), the use of a newly-proposed standard tracking model to exchange information between the serious game and the analytics platform, allowing reusable tracker components to be developed for each game engine or development platform; and 2), the use of standardized analysis and visualization assets to provide general but useful information for any SG that sends its data in the aforementioned format.

3 Measurify APIs

Measurify is a cloud-based Application Programming Interface (API) designed to support an efficient workflow for preparing different measurement-based data-rich applications, especially but not exclusively from IoT devices [Atmos]. The framework integrates APIs implementing Representational State Transfer (REST) services [12], that provide a platform-independent HTTP interface (e.g., [13]). Measurify exploits the open source MongoDB non relational database technology for data management, and on the NodeJS programming language, supporting by design seamless cross-platform portability, without being locked to a specific cloud vendor technology (e.g., Amazon Web Services for the Internet of Things (AWS IoT) [14] or Microsoft Azure cloud platform [15]), which is an important limit of current approaches. According to the RESTful programming paradigm, the API exposes a set of resources, A resource is an object with a type, associated data, relationships to other resources, and a set of methods that operate on it. Measurify defines a limited but powerful set of resource types (Table 1), that have been suited to create applications in international research projects in the automotive and health domains [16,17,18].

Table 1. Outlook of the measurify resources
Full size table

The first step of the workflow supported by Measurify consists of the domain modeling, where the field objects (i.e., the objects involved in the measurements) are mapped to the API’s resources. In this phase, the IoT application designer has to define features devices, tags, and constraints. In the configuration (or deployment) step, the above designed model resources are straightforwardly encoded in a.json file, which is POSTed to the framework APIs, so to create the Application Database (ADB) structure. In the regime phase, the framework manages the ADB, allowing (i) dynamic insertion/update of users, things, field measurements and computation requests; and (ii) retrieval of results in terms of things, measurements and computation outcomes. The ADB structure can be updated during the operation as well, by POSTing/PUTting/DELETEing features, devices, and tags. All these actions happen only through the exposed resource routes (i.e., standard functions through which objects can be accessed), with the well known advantages of the RESTful approach in terms of scalability, encapsulation, security, portability, platform independence, and clarity of terminology and operations.

In the next section, we present the extension of Measurify in the direction of the edge computing, namely the Edgine (Edge Engine) module.

4 Edgine

Measurify is a cloud-based system designed to host applications receiving measurements data through a REST API. According to the edge computing paradigm, computation is being moved towards the edge, in order to optimize exploitation of resources. Availability of computing capabilities on the edge, close to the field sensors implies the possibility to implement sensor hubs, typically represented by microcontrollers, that can be configured from remote in order to deliver the dynamically needed information.

This is the founding idea of Edgine, an edge system designed to support efficient development of integrated IoT applications. Edgine features an HTTP communication interface with the cloud (particularly with the Measurify APIs) in order to download configuration settings and scripts that are executed locally.

The system code consists of two main parts: an initial one and a loop one. In the start phase, which is implemented inside the setup() function, the Edgine software authenticates itself and connects to the APIs to download its configuration parameters and the list of scripts to be executed. Then, during the infinite cycle loop, the software continuously executes each assigned script in sequence, processing the data and sending them to the Measurify APIs.

Each device associated with Measurify is described by a JSON (JavaScript Object Notation) file which includes, among others, the feature field, describing the expected type of the measurements to be delivered, and the scripts field, including all the scripts to be downloaded and executed by the particular device. A script consists of a sequence of instructions that typically conclude with the delivery of the processed data to Measurify. The raw data processing instruction types currently include simple arithmetic functions, computation of simple statistic operations (e.g., min, max, mean, median, stdev), and accumulation of values in a (sliding) window.

A key design requirement for Edgine is platform independence, in order to make this paradigm widely available across edge devices. Thus, we strived to keep the system as independent as possible from the hardware. To that end, classes have been created to allow developers to switch from Windows/Linux/Mac PC platforms to Arduino through the use of C preprocessor macros.

The differences between the two main platform types concern the Internet connection. In the Arduino case, an automatic connection to a WiFi network (predefined in the code) is performed, and the system is also designed to perform a reconnection in case of signal loss. In the Edgine version for PC-type machines, on the other hand, the system waits until a network connection becomes available, before trying to perform the initial authentication operations. At runtime, data that cannot be sent due to lack of connectivity are accumulated in a buffer, waiting for a connection.

Up to now, Edgine has been used in some IoT applications, concerning the monitoring of environmental parameters such as temperature, humidity, light conditions [19].

5 Serious Game Application

While Edgine has a clear application to edge devices, such as microcontrollers, mobile phones and automotive electronic central units (ECUs), its concept is abstract and can be applied to any device processing raw data. Thus, the Edgine can also be employed in a serious game with a goal to get measurements from it and pre-process them according to some basic scripts. Everything is dynamically programmed from remote. Conceptually, this corresponds to applying the IoT sensor measurement concept not only to reality but also to a game’s virtual reality.

To achieve this goal, it is necessary to insert the Edgine system inside the target serious game. This can be achieved by integrating Edgine inside a game engine. This is a general solution, that could be seamlessly applied to any game implemented with a given game engine.

For the case of Unity 3D, a widely used game engine, the idea is to develop a plug-in that would wrap Edgine, making it available to any game. For instance, it would be possible to configure the Edgine so to send to the measurement API (i.e., the generic analytics engine) information such as: the average distance covered by a virtual character in the last minute, the number of collected objects, the number of correctly answered quizzes, etc. The key advantage – especially in a software engineering perspective of efficient development - is the abstract and generic architectural approach, which makes the solution generally applicable.

In the following, we describe the implementation of Edgine in Unity 3D. In this game engine, C# scripts are usually employed to create components and specify their behaviour. Scripts can exploit libraries of additional functionalities, made available in Unity as plug-ins. Plug-ins are platform-specific native code libraries that can access operating system functions and other third-party libraries, that would not otherwise be available to Unity developers.

A Unity plug-in is equivalent Dynamic-Link Library (DLL), and can be efficiently developed with Visual Studio, which is quite similar to the Unity Integrated Development Environment (IDE). In the DLL, we implemented a single class, which wraps the methods intended to be exposed to Unity C# scripts. Inside the header, it is necessary to define such functions with the declspec(dllexport) attribute. Moreover, through the extern “C” label, it is indicated to the compiler that the C linking conventions should be employed for such function. This is necessary for a correct export, because, otherwise, C++ compilers would perform the mangling process, making them unreachable from Unity. The mangling is a technique used to distinguish functions that have the same name (overloading). The technique manipulates the name during compilation, so that it becomes unique. But this would make the final name unknown to the Unity developer. The C language, on the other hand, does not support function overloading, thus does not perform the mangling. Thus, the binary file generated by the compilation process contains the original method names. However, since the C linking convention is adopted, it is necessary that the input and output data types are those of the C, even if the body of the function is written in C++.

According to the Edgine programming model presented in the previous section, the class exposes two methods: Setup() and Action(). The first one will be used at start-up, the second one will be employed inside the game engine’s update loop. For the Unity implementation, we let that the service descriptive features (device, thing, feature, username e password) can be specified by the Unity user as parameters of the two exposed functions and are not hard-wired in the source code, which is the approach employed in the edge/embedded environment. Intuitively, the user can specify the names of the resources related to a correct delivery of measurements to Measurify. The Action() function, takes in input the name of the feature and the measurement value, which will need to be delivered to the cloud.

The compiled Edgine project produces five dll files: NativeCppLibrary.dll, pcred.dll, PocoFoundationd.dll, PocoNetd.dll, zlibd1.dll, of which the first one is the dynamic library containing the two functions mentioned above. In order to use the plug-in inside Unity, it is necessary to copy all these .dll files in the project directory.

As a proof of concept, we implemented a very simple application example of collection of data from a game scene and their delivery to the cloud. Particularly, we tracked the collisions of a ball with the borders of a squared region (Fig. 1). Every time a collision is detected with the wall, a variable is incremented. When the number of collisions becomes a multiple of 5, the value, added with the time stamp, is sent to the Measurify APIs, from where it can be manipulated, also in real-time, by other applications and/or user interfaces.

Fig. 1.
figure 1

Simple game scene in Unity3D

Full size image

This goal is achieved by following the standard steps of the workflow of an Edgine-extended Measurify application. The first step consists in POSTing to Measurify (e.g., through a common collaboration platform for API development, such as Postman) the names and, particularly for the Script resource, the contents of the resources needed to configure the environment so that it will be able to receive measurements from our application. In our case, the names are reported in the second column of Table 2. The Measurify APIs will create the corresponding resources, that will be available at the URLs in the third column, where the base url is http://students.atmosphere.tools/

Table 2. Measurify resources for the example application
Full size table

The next step concerns the development of the C# script that detects the collision and exploits the Edgine plug-in to transmit the corresponding value to Measurify. As anticipated, for the sake of flexibility, in the Unity implementation the Setup() and Action() functions are parametric, thus we allow the Unity user to specify through the Inspector module (Fig. 2) the values that will be passed to such functions.

Fig. 2.
figure 2

Variables set by the user inside the Unity Inspector

Full size image

The Unity scripts are organized in a structure that strictly corresponds to the Edgine programming model, with an initial Start() method, executed only once at the beginning of the programme, and an Update() method, which is continuously called, at each frame’s update. The mapping for an Edgine application in Unity is thus straightforward: the Setup() method is called inside the Start(), while Action() inside Update(), (Fig. 3).

Fig. 3.
figure 3

Start and update methods in the Unity script

Full size image

Start() accomplishes the task of initializing all the StringBuilder public variables involved in the script. Once such user-defined parameters are collected, the plug-in’s Setup() is executed, which performs the authentication in Measurify, getting the Json Web Token (JWT) that will be used in all the subsequent accesses to the cloud APIs. A Queue<Thread> object is also initialized, that will be useful during the game loop. The creation of a Thread object, in fact, is necessary when sending data to the cloud, in order not to block the continuous game update cycle. The threads are inserted in a queue to simplify their management, as they are created, executed once at a time and destroyed in order, making use of the Enqueue(), Peek() and Dequeue() methods.

The Update() method first makes a check on the number of accumulated collisions (a measurement is sent to the cloud only when a multiple of 5 is hit). Then, if the sending condition is met, a thread is created, consisting of the Play() function calling the plugin’s Action(), and added to the queue. Once created, a thread is ready to be executed as soon as no other thread is still alive. In this way, the application is not overloaded with threads. If some threads are still in the queue when the game is completed and quitted by the player, the OnApplicationQuit() method launches the missing threads before the actual end of the program execution. Finally, Fig. 4 shows an example of measurement received by Measurify in JSON format.

Fig. 4.
figure 4

The measurement received from the Unity game at the Measurify’s end

Full size image

6 Conclusions and Future Work

This paper has proposed the use of a new data toolchain for serious games analytics. The toolchain relies on the open source Measurify IoT framework for measurement management, and particularly takes advantage of its edge computing extension (Edgine), which can be seamlessly deployed cross-platform on embedded devices and PCs as well. The Edgine is programmed to download from Measurify a set of scripts, that are periodically executed so to get data from sensors, pre-process them and send the extracted information to the Measurify APIs.

Thanks to its powerful abstractions, this model can be seamlessly employed also in the virtual reality and gaming domain. Virtual sensors can be built in game engine scripts. This paper has presented the implementation of the plug-in which deploys Edgine in Unity 3D, allowing an easy delivery of virtual sensor information to Measurify. Just as a proof of concept, we have presented the utilization of the whole chain within a trivial game scene. We plan to employ and test the system with more complex serious game analytics, and this opportunity is possible for every researcher and practitioner thanks to the open source release of the whole Measurify framework [20].

There is another aspect beside the general applicability of the toolchain to any virtual sensors. In fact, the same software engineering approach (and actual Edgine tool) could be used also for measuring real world parameters, connected to the game. This opportunity is particularly relevant to augmented reality games and reality enhanced serious games (e.g., [21,22,23]). Moreover, the same system can be used also the physiological parameters of the player. To the best of our knowledge, no tool exists that is able to process information from such heterogeneous sources as physical and virtual sensors. For instance, the Edgine toolchain could be used to simultaneously get data about a game session and a player’s physiological status.