1 Introduction

Three-dimensional (3D) technologies, based either on tangible perception, such as the case of 3D printing, or on intangible perception, such as augmented reality (AR), are currently mature enough to be accessibly and efficiently applied and put to advantage in the field of education. Combining 3D technologies with information and communication technologies (ICT) increases the flexibility of their on-site or remote access and use. We are thus part of an ecosystem with optimal conditions for advancing teaching and learning through the development of contents that leverage resources available to us. In this regard, there is a great international exploratory momentum with the development of several research projects, which is also beginning to grow in Portugal.

This chapter aims at mapping the evolution of 3D technology, particularly in the application of AR and 3D contents to teaching, and presents a synthesis of practical cases. Therefore, it begins by setting the context for the insertion of (intangible) three-dimensional technologies in the various sectors of society. Then, some international and Portuguese works are described which are being developed by applying 3D technologies, mainly in education and teaching. To this end, and in line with Cardoso et al. (2007, 2010, 2013), a thorough literature research was conducted, with particular focus on recent years (between 2010 and 2014, with regard to the application to education), although the technological framework includes some facts and milestones from the last 80 years. Finally, part of the research work that is being done on intangible three-dimensional technologies, in particular the implementation of AR, is described, as well as how these technologies can be capitalized on on-site and remote educational situations.

For this study, Google was searched using keywords and Boolean markers, including terms such as “ augmented reality,” “tridimensional,” “teaching,” “mathematics,” and “m-learning.” Using the same descriptors, the literature search was complemented with a more focused search on b-on, the Online Knowledge Library, where the journals Computers and Education, Computers in Human Behavior, Journal of Systems and Software, Computer Science, Advances in Engineering Software, and Social Behavior Sciences were consulted. The research was carried out mainly during 2013, with an update made until the start of the last quarter of 2014.

It should be noted that this study presents a chronological perspective of technological evolution of equipment and software, which currently translates into unique conditions for the effective implementation of three-dimensional technologies supported by AR and by ICT in the field of education. Therefore, the selected works, mainly papers, were conducted between 1997 and 2014 and reflect the developments from 1961 to the present.

In addition, the search was not restricted to research work in Portugal, in part because the first finding of the mapping is that there is a reduced amount of work in this field being implemented in this country.

A corpus of 33 references was therefore created, mostly composed of international peer-reviewed scientific journals, accessed via b-on, giving priority to papers that present a chronological evolution of augmented reality technologies, their relations with the evolution of ICT and, finally, to studies of three-dimensional content implementation of AR in higher education, in particular in mathematics courses.

2 Augmented Reality and Its Evolution

When talking about three-dimensional technologies, such as the ability to print three-dimensionally (3D printing), to capture 3D shapes (3D scanning), and to integrate virtual elements (AR) into the real world, it becomes apparent that 3D technologies advance alongside technological computer developments and are linked to the integration of ICT in our daily lives. An analysis of computer development will show some similarities to 3D technology development, as can be seen below.

Intangible content 3D technologies, such as AR, are closely linked to computer capacity, and calculation and thus their availability is related to the development of personal computers. AR is the integration of virtual images into the real world; this integration is performed using ICT. Reality is augmented with virtual elements: a mobile device with a camera, such as a tablet, a cell phone with Android or iOS operating system, or a computer, enables anyone to access content provided with AR. The development of AR contents has been following computer and ICT technological developments. Figure 1 presents a chronological summary of the main events that contributed to the current stage of development in these areas (see also Table 1).

Fig. 1
figure 1figure 1

Historical evolution of AR and technological evolution milestones in computers, ICT, and image display systems (Source: Data collected for this study)

Full size image
Table 1 Historical evolution of AR and technological evolution milestones in computers, ICT, and image display systems (Source: Data collected for this study)
Full size table

For example, in 1961, the cinematographer Morton Heilig registers the patent of an innovative system, the Sensorama, which allows the user to experience an immersive cinematic session. In 1968, Ivan Sutherland develops a computer-controlled immersive helmet (Carmigniani et al. 2011; Sutherland 1968). This was the first virtual reality and AR system; it was developed at Harvard University in Utah, USA. In the 1970s, this technology became known as Artificial Reality (Zhao 2009). Later, Myron Kruger, at the University of Connecticut, develops the Videoplace, which consists of a living room for human-computer interaction. This system uses information from a camera that is transmitted to a computer and then projected on a screen, allowing interaction with Artificial Reality (Nagler 1994).

In 1993, Feiner and collaborators publish the first paper in the field of AR. The application they developed, named KARMA, was validated in technical staff training for printer maintenance. Two years earlier, Feiner et al. (1993) had concluded that AR-based content would have an important role in training and that increasing its use only required a reduction in display system sizes and a more flexible usability. Indeed, viewing/display systems were relatively large, inflexible, and uncomfortable 23 years ago. Research and development into this new perspective of the real world continued, mainly in military applications of NASA (National Aeronautics and Space Administration) for the training of pilots, astronauts, military, and in the medical field.

Many of the innovations and developments in Artificial Reality never came to be applied successfully until Zimmerman and Lanier introduced the “Dataglove,” a more user-friendly way to interact with the virtual environment (Sturman and Zeltzer 1994), in which users could manipulate objects in the virtual world through a glove. At that time, in 1989, Jaron Lanier coined the term “virtual reality” and was the first to bring the technology to the public, selling “datagloves” as a way into the virtual world (Zhao 2009).

Finally, in 1990, the designation “augmented reality” is attributed to Tom Caudell, who created the term while working for Boeing to develop a head-mounted display system to help with wiring instructions for planes via the projection of plane schematics on boards at the factory (Vaughan-Nichols 2009). In 1994, this juxtaposition of virtual and real objects is explained by Milgram et al. as a continuum between real and virtual environments (see also Milgram 2006), as can be seen in Fig. 2.

Fig. 2
figure 2figure 2

Virtuality-Reality continuum (Source: Milgram et al. 1994)

Full size image

In 1997, Ronald Azuma wrote a report defining the scope of AR and listed three important criteria to define it, separating AR from “artificial and virtual realities.” Azuma’s three criteria to define AR are: it combines the real and the virtual, interactive in real-time, and registered in 3D. From this moment, the development of AR grew, and in 2002 the first AR outdoor game, called “Quake,” was presented (Thomas et al. 2002). In 2005, the Horizon report describes AR as a key technology for the 5-year development of applications. In fact, the popularity and the development of smartphones have brought AR to users. At that time, two different forms of AR were identified and defined: one based on location and another based on markers or patterns. In the case of location-based AR, the use of the mobile’s GPS determines the location, and thus a layer of information is added on top of what we are seeing with the camera (Billinghurst 2011; Carmigniani et al. 2011). Two examples of location-based AR applications are: the Wikitude application, released in 2008, and the Layar application, released in 2009. Both are available for mobile devices.

In the case of AR based on marks or patterns, the mobile device (mobile phone, tablet, or laptop camera) recognizes markers that give information or three-dimensional elements. The HITLab in New Zealand was the first to produce markers in AR printed in newspapers, serving, for example, as a means of advertising for Wellington Zoo. As soon as readers registered the marker on their mobile device, a 3D animal appeared on the page (Schmalstieg et al. 2011). Another example of technological advancement is the Google’s glass, which was launched in 2012 for testing and released to the public in 2013, when they began to be marketed in the United States. Although the patent has been registered by Google, other applications have already been developed by other companies and are emerging and preparing to appear on the market.

The incorporation of virtual elements into the real world now gains a new dimension. One example is the integration of technologies such as AR and three-dimensional noncontact scanning, which enabled Jeff Powers and his team to develop a system which, in addition to allow viewing the real world through the “window” of a tablet, can scan the same real world so that the geometric boundaries of real-world elements are recognized in real time by overlapping AR virtual elements. An example of the integration of noncontact scanning technology with virtual content, namely, a sensor visual field scan identifying objects and their forms, can be found in http://structure.io, whereas an example of an application of the integration of noncontact scanning technology with virtual content can be seen on video on the Internet in http://youtu.be/39v5OoBJFDk. This new feature opens an even greater field of applications and ways of displaying contents. It represents the interaction between a set of balls that fall on a bench and its surroundings (Structure sensor 2014). In this case, the effect of gravity and of object borders is apparent. Many different applications are being developed with the expansion of ICT and accessibility to mobile devices. The potential application areas of AR cross all subjects. Several examples of these applications are presented below, especially those in education.

3 Augmented Reality in Education

Three-dimensional technologies are still at a very early stage of application in teaching and learning. Nonetheless, there are several fields of knowledge in which they have been recently implemented and studied by several authors (among others: Bujak et al. 2013; Fonseca et al. 2014; Kamarainen et al. 2013; Wojciechowski 2013; Wu et al. 2013; Di Serio et al. 2013; Martin-Gutierrez et al. 2012; Nee et al. 2012; Kaufmann and Schmalstieg 2003). However, in order to have a historical and broader perspective of the application of AR contents and technologies in education, training, and teaching, some studies and applications from the last 20 years are presented. The abovementioned case of Feiner and collaborators is resumed: as stated, in 1993, they implemented the application KARMA to speed up the training process in laser printer maintenance. In the late 1990s, Kancherla et al. (1995) described the application of virtual reality and AR technologies to the kinematic and dynamic analysis of body motion applied to anatomy seminars.

In 1997, Inkpen presented a study in which specific contents were developed to stimulate learning via a computer. These contents were not developed in AR. However, they were precursors in analyzing the effect of technology-based learning. In addition to the specific development of applications and software to stimulate learning, Inkpen (1997) examined the possibility of simultaneously working on computers with two mice. The results showed that motivation and learning were increased with group work in comparison with the individual use by each child. In 1993, Lu and collaborators presented a study at the opening session of the CIRP (Collège International pour la Recherche en Productique), a major worldwide engineering academy, in which he stressed the importance of AR in the development and manufacture of products.

In 2000, Weidenbach and his team developed a system with AR content for the medical field, in particular for training two-dimensional echocardiography analysis. One year later, Taxen and his collaborators developed virtual environments by immersing an AVATAR (graphical self-representation of the Internet user intended for virtual environments) to teach mathematics, in the context of learning content. Still in 2001, Billinghurst also developed mathematics content displayed in AR under the name “MagicBook.” In 2003, still in mathematics, Kaufmann and collaborators described the implementation of the “Construct3D” system. This allowed an evaluation of the importance and flexibility of AR even in collaborative environments, as well as underlining the importance of such environments in student-student and student-teacher interaction. This system consists of three-dimensional mathematical content supported by display equipment and collaborative work, either on-site or remote – see Fig. 3a and b, respectively.

Fig. 3
figure 3figure 3

Construct3D System for collaborative learning (a) face-to-face and (b) remote (Source: Kaufmann and Schmalstieg 2003)

Full size image

In other areas, and still following a chronological evolution, Liarokapis et al. (2004) and Nee et al. (2012) studied the implementation of AR in manufacturing projects and processes in engineering. Moreover, Quirós et al. (2008) and Maier et al. (2009) developed chemistry-oriented applications, thus enhancing the visualization of atoms and molecules as well as that of chemical reactions. Martin-Gutierrez et al. (2012) studied the application of AR in teaching and in the spatial perception of mechanical engineering students. Later, in 2012, they studied the applicability of AR to electrical technical engineering, while Fonseca et al. (2014) examined its applicability to architecture. In the same year, Salinas et al. (2013) developed specific software for three-dimensional modeling of mathematical functions and conducted a study highlighting the important role of these technologies in group motivation and in reinforcing collaborative work.

Some interesting advantages of AR application were observed in the works that were analyzed. In the study of Martin-Gutierrez et al. (2012), the increase in students’ self-learning ability was highlighted, giving teachers more time to focus on explaining more complex issues. The study of Fonseca et al. (2014) outlines the advantages of AR tools for increasing spatial perception, providing in situ views of hypothetical scenarios for future construction, thus allowing an exploration and analysis of several solutions.

In an earlier study on m-learning systems, Ismail et al. (2010) describe high user satisfaction with additional mobile learning tools. The users felt supported and motivated to use mobile applications with an accessible language. Indeed, systems usually used in m-learning, such as these mobile communication systems, can enhance field observations and explorations when including AR content, because observed reality can be explained with the (augmented) addition of virtual content (explanatory videos, schematics, and three-dimensional designs, among others). This interaction contributes to greater autonomy in the learning process. It should be borne in mind that content made available and developed using AR technology can be accessed anywhere from m-learning support systems, such as mobile phones or tablets.

AR enables the development of conventional contents (e.g., books, lecture notes, presentations) but adds specifically programmed graphics which are recognized by an AR application and then activate additional explanations when displayed (such as three-dimensional, files explanatory videos and/or images). Figure 4 represents an example developed in AR to support the teaching of mathematics.

Fig. 4
figure 4figure 4

Three-dimensional representation of a hyperbola by intersection of a plane in a geometric solid (Source: Centre for Rapid and Sustainable Product Development (CDRSP)©)

Full size image

The contents of teaching and learning, in this particular case of mathematics, can be designed as usual based on an explanation on paper and complemented with a description of equations based on two-dimensional images. Content such as 3D files, videos, and explanations of intermediate steps can be added to these standard elements. This provides an integration between a traditional mode of visualization of content on paper and the use of a complementary AR technology.

As mentioned above, AR facilitates the integration between the real world and the virtual world, allowing the simulation and visualization of contexts and situations that could not be implemented otherwise. There are many areas of study and learning in which AR technology can be useful. Furthermore, AR technology brings a significant added value to areas that involve hands-on and experimental practice, such as science and engineering courses. In addition to the integration between the real world and virtual contents in the classroom, it is also possible to create contents combining several other environments. In addition to the most common environments (at home, at the office, in the living room), AR facilitates the development of contents in various contexts and environments for each individual. This enhances the interaction between in situ observation of the real world and the addition of theoretical and explanatory contents. The flexibility of AR tools allows for further experimentation and exploration of the real world by introducing real-time virtual explanations. One example is the project “EcoMobile,” described by Kamarainen et al. (2013). This project assessed the implementation of AR to learning by using mobile devices in contexts in which students are exposed to real situations. In this case study, the influence of AR content was assessed during study visits, and it was concluded that students showed an increase in interpretation flexibility when exposed to real situations and obtained explanations about their actual, real-time observation. In this way, learning is focused on the individual and each one can have access to explanations and support in the form of differentiated AR content at the moment that learning takes place.

4 Future Directions

The chronological-historical mapping above shows that AR, as well as mobile learning, is a breeding ground for education. The mapping also shows that application of AR to educational situations has benefited from technological development, in particular mobile devices and m-learning. Therefore, it is believed that three-dimensional technologies, more specifically intangible technologies (such as AR), will continue to be included on the educational agenda. Finally, part of the joint work which is being developed in mathematics in higher education is presented. For example, this work aims at assessing content produced through three-dimensional, tangible (3D printing), and intangible (AR) technologies. This work is included in an ongoing project at the Portuguese Open University and at the Polytechnic Institute of Leiria, which will result, for example, in a PhD thesis in education, in the specialty of distance education and e-learning.

The broader, international, ongoing project, 3D4εDU – Three-Dimensional Interactive Contents in Higher Education, is a strong challenge and a strong commitment to create three-dimensional content that can be democratized and made accessible. Besides including intangible 3D content accessible through AR, the project also focuses on the democratization of low-cost 3D printing to provide blind students with a three-dimensional perception of even more abstract concepts like mathematical functions. This is an example of inclusion which could take place in other sectors of society. In general, contact with new technologies during the course of a student’s training will allow them to learn to apply similar technologies to different sectors and moments in their future professional and personal life, in a lifelong learning rationale.

The 3D4εDU project intends to develop contents in other areas beyond mathematics, such as chemistry, life sciences, and optoelectronics. Furthermore, the ongoing work in mathematics in Portuguese higher education will be extended to other disciplinary and international situations, with the involvement of researchers and collaborators from Brazil, China, England, India, and Thailand. Involving different intervention areas and locations will boost the internationalization of teaching in Portugal in terms of defining, implementing, and assessing three-dimensional contents applied to learning.

The abovementioned innovative research work (three-dimensional technologies as a contribution to the learning of mathematics in higher education), which includes an experimental and analytical branch, has been further developed up to the design-based research (DBR) current second iteration of the implementation of three-dimensional content. This means that in the first semester of the academic year 2014–2015, a system is being implemented to provide 3D AR content in the cloud (unlike tangible content, which can be provided in the classroom). This implementation was subject to previous validation, both remotely and on site, during an exploratory phase which took place in the second semester of the academic year 2013–2014. Professors of mathematical analysis (AM) of the Technology and Management School (ESTG) of the Polytechnic Institute of Leiria (IPL), Portugal, were involved in the creation of 3D AR content which has been used to supplement mathematics teaching, particularly in engineering courses. Figure 5 illustrate a validation moment in the classroom during the first DBR iteration. There were technical constraints that are expected to be solved, but which did not compromise the excellent reactivity of students who accessed and tested this 3D AR content.

Fig. 5
figure 5figure 5

(a, b) Implementation of 3D AR content in a mathematical analysis seminar of the ESTG-IPL (Source: Teresa Coimbra©)

Full size image

In previous studies supporting the current one, projects were specifically developed to introduce new technologies and content at different levels of vocational education and training (primary education, secondary education, higher education, continuous training, and advanced training). The results were evaluated and their advantages and disadvantages carefully analyzed to define best practices for a proper implementation. Therefore, contents were developed in the Metaio application, which can be accessed through the Junaio application, available on several mobile devices. These contents have been created for an exploratory analysis of the visualization of medicine and biology elements (see Figs. 6 and 7).

Fig. 6
figure 6figure 6

Three-dimensional representation of a human heart (Source: CDRSP©)

Full size image
Fig. 7
figure 7figure 7

Three-dimensional representation of the DNA double helix (Source: CDRSP©)

Full size image

These two case studies show that AR technologies can integrate both theoretical knowledge into real situations and real contexts into theoretical presentation forms. Bringing together and integrating both information formats can provide important leverage if contents are appropriately developed. In short, the integration and use of AR-based applications for content development, including formal access, is an asset which it is expected to be confirmed in mathematics, as stated above, at a time which is conducive to the democratization of, increasingly portable, more personal and more social technology.

5 Cross-References