Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

In past decade, many researchers in the educational engineering area have been studying focusing on ubiquitous theme. For example, CSUL (Computer Supported Ubiquitous Learning) or context aware ubiquitous learning (u-learning) have been constructed using computing technologies such as mobile devices, QR-code, RFID tag and wireless sensor networks [10, 13]. CSUL takes place in a variety of learning spaces, e.g. classroom, home and museum. Also, it provides the right information using the contextual data like location, surrounding objects and temperature.

There are such language learning and learning of nature science on application domains of CSUL. For example, Hwang et al. [14] reported about their ubiquitous learning system that students can study butterfly ecology, using PDA (Personal Digital Assist) at an elementary school nature science course in Taiwan. In their developed system, the learners can developed a concept map based on what they have learned from text book. Also, they can revise and review their concept map using PDA in the field.

On the other hand, Ogata et al. [11] reported about their ubiquitous learning system called SCROLL (System for Capturing and Reminding of Learning Log) that allows learners to log their learning experiences with photos, audios, videos, location, QR-code, RFID tag, and sensor data and so on, and reuse them with others. The goals of SCROLL are lying in helping users to easily record their learning experiences and recall them via the context, recommending other learners’ learning experiences for them, finding out individuals’ learning habits and supporting their learning accordance with personal learning habits.

These learning dataset in the ubiquitous learning system include spatiotemporal data. Spatiotemporal data usually contain the states of an object, an event or a position in space over a period of time. In the spatiotemporal data mining, they explored many challenges in representing, processing, analyzing and mining of dataset in spite of complex structures of spatiotemporal objects and the relationships among them [3–5].

Similarly, many issues have been raised about relationships between the learners and the ubiquitous learning logs in SCROLL. Their challenge is significant because it is important for learners to recognize what and how they have learned by analyzing and visualizing, so that they can improve their way of learning [13]. Mouri et al. [8, 9] proposed an innovative visualization system which integrates network visualization technologies and time-map in order to visualize complex relationships among them. Consequently, their system succeeded to reveal relationships between the learners and ubiquitous learning logs.

2 Related Works

2.1 Mobile CSCL and Context-Aware Ubiquitous Learning

Researches on the educational designs to enhance communication skills and to increase learning opportunities with mobile technologies are among the most popular topics in the researches on mCSCL (mobile Computer Supported Collaborative Learning). Consequently, it is expected facilitating their interactions and feedbacks among students or teacher and students.

For example, Gustavo et al. [2] constructed a mobile learning environment based on the theory of constructivist education, and succeeded to maintain students’ motivation and making students increase active social interactions. In their experiment they assigned 7-year-olds with the tasks to learn alphabets and syllables in the group learning style. They compared the two learning modes: Syllable-mCSCL (collaborative learning using mobile device) and Syllable-CL (without mobile device). More social interactions among the participants were observed in Syllable-mCSCL mode, while, lack of the motivation and interest was observed in Syllable-CL mode, which were regarded to be serious problem in learning process.

Similarly, Wong et al. [16] reported a collaborative learning aimed to form Chinese character component based on mCSCL. The objective of their study is to reinforce rules of orthography to students. They found awareness patterns of orthography among students’ social interactions. However, in these mCSCL studies, the researchers did not take into consideration the methods for visualizing the linking relationships between location and time.

Researchers on context-aware ubiquitous learning, on the other hand, have been constructing ubiquitous learning environments where we can study anywhere and anytime. They integrated knowledge and location information by using cutting-edge technologies such as RFID-tag, QR-code, NFC-tag and GPS.

For example, Hwang et al. [13] attempted to guide their learners towards the optimal learning path by using mobile device and the RFID-tag attached to plants. If students arrived at the target plant by using their system, the system will give some questions to the learner. For example, if students do not know the plant name, the system will ask the plant features (shapes, color, trunk and so on). By answering them in accordance with the questions, the system will present hints or candidate items of the plant. Consequently, students succeeded to deepen their knowledge of plants..

In contrast, Li et al. [6] accumulated all the location information to the database on the server by using GPS sensor. The accumulated data to the database include not only the data collected in one evaluation experiment but also those collected by a number of the research studies for a long period of the SCROLL project (2011 ~ 2014) [12]. At the present stage, there are not enough researches on visualization of relationships among knowledge, location and time from a large amount of raw ubiquitous learning logs. As one of the reasons, it has been pointed out that it is difficult for most mobile learning systems to accumulate learning logs for a long period.

To tackle these issues, this paper describes innovative visualization methods in order to reveal them. It allows us to reveal not only the relationships among knowledge obtained by learners but also the relationships between knowledge and locations, and knowledge and time by using visualizing relationships among ubiquitous learning logs in SCROLL.

2.2 Learning Analytics

In recent years, with the spreads of LMS (Learning Management System) and CMS (Course Management System), Learning Analytics (LA) analyzing various evidences in the education and learning has been drawing an attention [15]. It is expected enhancing the quality of education by detecting effective and efficient learning information because these histories are reflecting on learners’ activities.

To date, LMS and LMS enabled us to record learners’ access logs onto server. The LA aims for practical use based on learning mechanisms revealed by visualizing, mining and analyzing vast amount of learning data. This paper focuses on the SLA (Social Learning Analytics), a subset of the LAK (Learning Analytics and Knowledge) [1]. The SLA puts forward presenting appropriate information to learners at the appropriate timing through the dashboard in real time.

Therefore, this paper aims to reveal about relationships between learners and ubiquitous learning logs on spatiotemporal fields. It is expected to contribute to educational improvement and strategies below;

  1. (1)

    This study facilitates the analysis of learners by visualizing all data on spatiotemporal.

This study enables future prediction about learners and learning environment from visualized learning logs.

3 SCROLL

3.1 Design

In order to support such formal note taking and reminding, we designed and implemented our ubiquitous learning system called SCROLL. One of the objectives of SCROLL is to support international students in Japan to learn Japanese language from what they have learned formal and informal setting. It adopts an approach of sharing user created contents among users and is constructed based on a LORE (Log-Organize-Recall-Evaluate) model [11].

In the “Log” process, when learners face problems in daily life, they may learn some knowledge by themselves, or ask others for help. The system records what s/he learned during this process as a ULL. In the “Organize” process, when a learner tries to add a ULL, the system compares it with other ULL, categorizes it and shows similar ULL if they exist. By matching similar objects, the knowledge structure can be regulated and organized. In the “Recall” process, the learner may forget what s/he has learned before, Rehearsal and practice in the same context or others in idle moments can help the learner to recall past ULL and to shift them from short-term memory to long-term one. Therefore, the system assigns some quizzes and reminds the learner of her/his past ULL. In the “Evaluate” process, it is important to recognize what and how s/he has learned by analyzing the past ULL, so that the learner can improve what and how to learn in future. Therefore, the system refines and adapts the organization of the ULL based on the learner’s evaluation and reflection. All the above learning processes can be supported by SCROLL.

3.2 System Interface and the Learning Scenario

SCROLL mainly focuses on language learning field. One of typical scenario of its use is to assist international students to study Japanese in Japan. In this case, Japanese language learners, who face rich learning contexts every day, can gain much knowledge from their daily lives in different kinds of situations, such as shopping in the market, seeing a doctor in the hospital, having a haircut in a barbershop, visiting the museum and so on.

The learners can record those situations and their experiences as ULL with a photo using desktop PC or mobile device and SCROLL as shown in Fig. 1.

Fig. 1.
figure 1

Adding a ubiquitous learning log

Full size image

Learners can reflect what they have learned using this interface anytime and anywhere. The learning log includes author name, language, created time, location (latitude and longitude), learning place and tag. The system will categorize automatically created time and learning place to each attribute. For example, if a learner learned an envelope at the Fukuoka post office (Fukuoka is the name of a prefecture in Japan), the system will add automatically “Post office” of the attribute name to database.

The learners can review past ULL by using quiz function as shown in Fig. 2. There are three types of quizzes generated automatically by the system, which are yes/no quiz, text multiple-choice quiz and image multiple-choice quiz. Figure 2 shows an image multiple-choice quiz interface generated automatically based on the meta-data of ubiquitous learning logs.

Fig. 2.
figure 2

Quiz interface

Full size image

The learner learns an envelope at the post office and he saves the knowledge as learning log on SCROLL server. After that, there are cases for the quiz function to handle with the learning log [6]:

  1. (1)

    Recall via context: When the learner enters the post office again, the system will provide him with reminder quizzes in order to support him recall the envelope.

  2. (2)

    Study when you prefer: If the system finds that the learner has a learning habit that he usually studies at home in the evening. If the system detects that it is evening and the learner is at home, the system will prompt him to review what he learned.

  3. (3)

    Learn from other: If another learner enters the post office and she has the same language ability with the previous learner, the system will recommend the learning log about the envelope for her.

Learning Log Dashboard (L2D) in SCROLL shows these quiz histories and their learning logs histories. L2D is to enable learners to reflect on their own activities and to reinforce what they have learned [7]. L2D focuses on statistical data on every learner’s usage of the system. L2D shows the number of learning logs that a learner uploaded and the number of completed quizzes, as well as memorized learning logs and incorrect answer of the quizzes. It is easy to grasp incorrect answers on a word and to control in the dashboard.

In order to reveal relationships between learners and ubiquitous learning logs, the visualization and analysis engine is implemented. In the next Sect. 4, this paper will introduce visualization system for analyzing ubiquitous learning logs.

4 Visualization System for Analyzing Ubiquitous Learning Logs

4.1 Structure Based on Network Graph in SCROLL

To visualize and analyze several relationships between the learners and the ubiquitous learning logs, we have uniquely defined them as three-layers structures as shown in Fig. 3.

Fig. 3.
figure 3

Three-layer structure in SCROLL

Full size image

The upper layer contains each author in order to confirm position of own or other learners. For example, if a learner learned various ubiquitous learning logs on SCROLL, there is a possibility that other learners had already learned it. Therefore, when the learner learned them, they can grasp other learners’ status with past learning experiences. That is, they can know “who should we ask the knowledge”.

The intermediate layer contains the knowledge that learners learned. Also, some fields of learning tasks can be included in this layer. For example, some task-based learning in ubiquitous learning environment can be carried out using knowledge and event. The scalability of the layers can be enhanced and the field of visualization can be widened by linking one’s own learning logs to the knowledge learned by doing tasks.

The lowest layer contains data such as location and time. The layer allows the learners to grasp when and where they have learned by revealing place and time.

Analysis by categorizing three-layers has following advantages:

  1. (1)

    Places with a large number of links to the related knowledge are the places where they can learn a lot of knowledge. For example, if a certain supermarket or convenience is related with a lot of knowledge such as natto (a traditional Japanese food made from fermented soybeans), green soy beans, tofu, miso soup, and cup noodle, by analyzing relationships between the knowledge and the location. The System can provide learners with a valuable learning information.

  2. (2)

    Knowledge which is related to many places is the knowledge which we can learn in various places. For example, if a learner experience tea ceremony of a traditional Japanese culture at the university in Japan, a set of tea ceremony related knowledge (e.g. tea, seize: to sit in the correct manner on a Japanese tatami mat) can be learned in other various places. The tea can be learned by purchasing at the supermarket and the constellation (seiza in Japanese) can be learned at the martial art gym.

4.2 The Layout Types of the Network Graph and the Color of Visualized Nodes

The layout consists of using the original layout we have developed as shown in Fig. 4. The original layout will be categorized four areas. The knowledge centered on collocational network is shown in time-series order what they have learned. Similarly, the spatial collocational network is visualized the place linking each other, and the temporal collocational network is visualized the time linking each other.

Fig. 4.
figure 4

Original layout

Full size image

The learners might get confused when they recognize past learning logs because there might be too many of visualized nodes. Therefore, it is definitely necessary to establish some criteria for distinction of each node. To effectively distinguish kind of each node, we defined as below using node color (Table 1).

Table 1. Color to distinguish the kind of nodes
Full size table
  1. (1)

    Pink color node shows the learner’s own name on the upper layer. If connecting the pink node to yellow node on the intermediate layer, edge color will be decided as pink so that they can be easily recognized as the learner’s own logs.

  2. (2)

    Blue color node shows the names of other learners on the upper layer. If connecting the blue node to yellow color node on the intermediate layer, edge color is decided blue color.

  3. (3)

    Green color node shows the names of veteran or famous learner on the upper layer. Ig connecting the yellow node to the green node on the intermediate layer, edge color will be decided as green color.

  4. (4)

    Yellow color node shows both the learner own knowledge and the knowledge of other learners. For example, the learner can recognize his own knowledge because edge between the learner own name on the upper layer and the knowledge on the intermediate layer is pink color. In addition, the learner might discover knowledge of other learners related to own knowledge.

  5. (5)

    White color node shows the location of the learners on the lowest layer. The node includes latitude, longitude, building names and the attributes.

4.3 Visualization Interface Combining Network Graph and Time-Map

Time-map is a library of javascript, which collaborated with Google map and SIMILE (Semantic Interoperability of Metadata and Information in unLike Environments) TimeLine. SIMILE focuses on developing robust, open source tools that empower users to access, manage, visualize and reuse digital assets. The time-map function means that the user can scroll the timeline and then the Google map will display the learning logs recorded during learners’ selected period. It is designed to help learners to reflect what they have learned.

For example, if a learner clicks his learning logs on timeline, Google map will display their positions as shown in Fig. 5(2). After visualizing log information, Time-map will facilitate learners to reflect on their logs with spatial and temporal information. They are able to grasp their learning context and time zone. Also, it is a possibility that the geographic information is a clue of recalling what they have learned.

Fig. 5.
figure 5

Visualization interface

Full size image

In this paper, the interface combining network graph as shown in Fig. 5(1) and Time-map as shown in Fig. 5(2) for visualizing relationships between the learners and ubiquitous learning logs is shown Fig. 5(3). It consists of the following component:

  1. (1)

    Search form: This input form is used to search target word (e.g. ‘natto’ and ‘tofu’) on the all networks of SCROLL.

  2. (2)

    Layout form: The learners will choose one layout in this select form (e.g. Random layout, Force-directed layout, Yifan multilevel layout and original layout).

  3. (3)

    Network graph: The network graph shows the layout calculated by the system, and the layout in Fig. 5(3) shows a sample of the original layout. Also, the network graph and time map function are linked each other. For example, if a learner clicked a certain node on the network graph, the time map will show the location and time corresponding to it. Therefore, learners can obtain its location and time information.

  4. (4)

    Time map: Time map function consists of the timeline and Google map. It represents the shift of learning history in accordance with lapse of time. Learners might forget their learning logs when and where they have learned before. Therefore, the system will remind them of their learning logs recorded during the specified period of time by showing them on the timeline (default: two month before and after the setting time). Besides, the system will lead them to be aware of knowledge recorded right before or after the knowledge of their interest which was recorded by other learners. Therefore, it will give them a clue on what to learn in the next learning step.

  5. (5)

    Analysis results (Knowledge): The analysis by knowledge is shown as a trend ranking in order to expand their learning opportunity. By arranging ULLs in the in-degree centrality order, they will know ULLs that they are likely to study in the next step. That way they are able to have more learning opportunity.

  6. (6)

    Analysis results (Place): Place analysis is based on locations where they have learned. The place of high importance means the location where there are a lot of opportunities to learn if visiting them. It is analyzed in the same way as in-degree centrality analysis, and the system shows the results to the learners.

5 Conclusion and Future Works

This paper describes ubiquitous learning system called SCROLL, and we introduced an innovative visualization system for analyzing ubiquitous learning logs. SCROLL allows language learners to geo-spatially tag vocabulary words they learn and update them to the system. The number of ULLs will keep increasing as the learners keep learning. By using the visualization system we described, it can be revealed relationships between learners and ubiquitous learning logs. Besides, the system will lead them to be aware of knowledge recorded right before or after the knowledge of their interest which was recorded by other learners, by utilizing network graph with time-map.

As future works, it is necessary to recommend and present past learning logs on the system in accordance with each learner’s condition detected from some results such as social analysis, association analysis and decision tree. In addition, it is also necessary to evaluate whether detected analysis results are appropriate or not.