1 Introduction

STEM is an acronym that refers to all teaching and learning fields of Science, Technology, Engineering and Mathematics. It typically includes educational activities and learning tasks across all grades (from primary to post-doctorate levels) in formal or informal activities. The majority of STEM courses depend upon an instructional process for the implementation of learning activities, with the intention of facilitating different laboratory experimental exercises. Practical hands-on experiments require not only higher-order, problem-solving, analytical and critical thinking skills that all students who participate in STEM courses need to have, but also instructional design frameworks and strategies used in a learning environment, in order to support the laboratory courses. This statement is crucial for the theoretical and practical research, with learning content generally focusing on the creation of an instructional environment and learning materials that will bring learners from the state of not being able to accomplish certain tasks to the state of having the ability to accomplish them. However, several limitations that usually appeared in real life settings as significant are the following (President’s Council of Advisors on Science and Technology, PCST 2012; Fiedler and Haruvy 2009; Uttal and Cohen 2012): (i) failure to realize experiments due to the limited spatial and temporal capacity, (ii) limited capacity in using available resources with a risk of injury for participants, (iii) the expensive equipment or materials cause inability to problem-solve in several activities, and (iv) the repetition of some experiments are significant and associated with higher levels of financial burden. All these drawbacks can create a sense of dissatisfaction for students’ participation in STEM courses. This situation cannot maximize the potential of students’ skills and their (socio-) cognitive development (Sanders 2009).

Due to the advancement of technology, instructors and educators in STEM courses have started to utilize digital environments so that configure well-defined activities with specific purposes, along with theoretical underpinnings of instructional design theories. More specifically, Potkonjak et al. (2016) have referred that by operating a virtual laboratory for STEM courses, each student must feel like working in a digital-oriented environment with the sense of utilizing real authentic devices. Starting from this general statement, the same authors proposed the following four criteria that should be formulated: (i) the graphical user interface for each piece of equipment must be identical to the corresponding real devices, (ii) the behavior of the virtual system (e.g. control variables) must be equivalent to the system behavior in the physical paradigm, (iii) the graphical user interface must provide what makes students feel like they are looking at a real authentic thing, and (iv) the laboratory digital-oriented space must be created in order to allow for communication and collaboration among students and with the instructor (or expert in the field).

Noteworthy are also the disparities in the introduction of two-dimensional (2-D) or three-dimensional (3-D) technologically-advanced learning environments. Additionally, successful closure of the gaps in achievement across the majority of STEM levels can be attributed to the effectiveness in real life experiments by using 2-D or 3-D digital-oriented environments that follow an instructional design framework underpinned by a theoretical foundation of learning for example such as Constructionism. Previous studies (Capraro and Slough 2013; National Foundation for Educational Research, NFER. 2010) have reported several difficulties in implementing learning activities for STEM education in practical hands-on experiments via 2-D digital-oriented environments. The utilization of these platforms is not so encouraging for students’ participation, because of the limitations that emerge during experimentation in laboratories towards a better understanding of how to (co-) construct or (co-) manipulate high-quality practical hands-on experiments tasks. As a result, students’ mental representations using innovative 2-D digital-oriented learning materials cannot offer the potential for better understanding conceptual concepts, theorems or general educational issues through hands-on tasks in digital settings (Potkonjak et al. 2016). Nevertheless, as candidate platforms for STEM courses cannot yet met all users’ (instructors and students) needs for participation in high-quality development courses. Therefore, fundamental issues that concern a growing body of literature (Hernandez et al. 2014; National Research Council 2014; Wilkerson and Haden 2014) are not only students’ engagement and participation, but also the effectiveness of practical hands-on experiments that should be provided from the beginning of Primary and Secondary education (middle and high school) until Higher education.

There is generally occurred a common belief that learning tasks in STEM education cannot always follow a specific instructional design method to ensure that each user (instructor or student) understands the management responsibilities that needs to have for knowledge acquisition. Also, during the instructional process, the learning outcomes do not always meet well-designed and developmental expectations (Han et al. 2014; Navruz et al. 2014). Unquestionably, throughout this event, instructional researchers, practitioners and educators need to build a strong case as to why engaging students in STEM courses using technologically-advanced environments is so important issue. This is an ongoing challenge that needs further investigation underpinned by theoretical foundation of learning theories (Davis 2012; Sanders 2009; Smith et al. 2009). In addressing the above challenges, two prominent and immediate needs referring the effectiveness of learning in STEM courses and their educational value emerge:

  1. (a)

    From a practical-didactic point of view, practical hands-on experiments should be delivered meaningfully by using tools that simulate, manipulate or implement learning scenarios in conditions that mimic/−or illustrate those of students’ real life situations delivered with instructional design methods or models (Davis 2012). Building upon students’ previous experiences, a learning process for knowledge acquisition is another critical challenge, since in STEM courses, they frequently try to explain and implement their solutions and justify the suitability of proposed solutions through complex ill-structured problem-solving activities. Such a learning process, requires an environment that can provide realistic learning conditions using a specific instructional design method (President’s Council of Advisors on Science and Technology, PCST 2012). Instructional design method is defined as a process of creating an instructional environment and learning materials that will bring learners from the state of not being able to accomplish certain tasks to the state of being able to accomplish them sufficiently (Smith and Ragan 2000). It includes firstly a taxonomy of cognitive, emotional and social course objectives, and secondly an identification of the learning material that students need to collect for the knowledge acquisition. In other words, a well-structured instructional design method combined with learning theories or models can assist instructors to clarify the appropriate conditions on how students can learn. The learning goals of the above can also reinforce students’ cognitive and meta-cognitive skills, which are obviously important and can further foster their engagement in the execution of learning activities (Wang 2013).

  2. (b)

    From a theoretical-didactic point of view, when innovative learning environments are utilized, different learning approaches or strategies must be implemented. Firstly, bolster students’ participation and enhance the learning outcomes along with using instructional design methods, and secondly the management responsibilities that should be available to all students in order to gain the new knowledge. Some of the most well-documented learning approaches for STEM courses are as follows (Davis 2012; Wang 2013): (i) project-based learning (PrBL), where students try to explore in a project several real- world problems through long-term or interdisciplinary or problem-based learning(PBL) approaches, in which students try to learn how to solve some problems in challenging tasks to acquire deeper knowledge; (ii) inquiry-based learning (IBL), wherein students learn as they search for information expanding their knowledge about a subject and perform with in-world activities, while pursuing some objectives; or (iii) digital game-based learning (DGBL), where learners are placed in a simulation or digital role-playing environment, in which the educational content is mixed with the challenges that they have to face during the game.

The above learning approaches have recently been combined with theoretical underpinnings based on learning theories that can be used for STEM courses. These are the following (Glancy and Moore 2013; Slough and Milam 2013): (i) Communal Constructionism (learning delivered when all students construct their knowledge domain, usually by constructing meaningful products, or by providing their findings in the environment and communicating with others to configure their actions); (ii) Constructivism (learning is a subjective and internal process of building meanings and it is considered as the result of organization and adaptation of new information into the existing knowledge that students already have to enhance their experiences); (iii) Social Constructivism (learning is achieved with the social interaction among students in scaffolding instruction, in which the instructor can give feedback on the on hand, and on the other should gradually abstains from students’ activities in contemplation of allowing them build upon their prior experiences and internalize new information to gain the new one), or (iv) Situated learning (learning is defined not as a function of an individual effort, but a socio-cultural function in a pre-defined framework carried out through communication and interaction among students who participate in ill-defined problem-based contexts). By combining theoretical foundations of learning theories with alternative learning techniques or approaches that referred above, instructors and scholars can implement learning scenarios to one or more of the following (Davis 2012; Pellas 2014): a) problem-solving processes using different forms of representation to present a solution to a problem, b) practical hands-on experiments for the experimentation and exploration of abstract concepts, c) collaborative learning processes by utilizing tools that enable students to apply the learning material towards proposed solutions.

By using web-based environments, instructional technologists (Khalil and Ebner 2015; Subbian 2013) have suggested the exploitation of educational resources in 2-D Learning Management System (LMS) and Massive Online Open Courses (MOOC), by utilizing mainly asynchronous communication channels, such as email or text-based messages. However, the digital-oriented (re-) presentations of knowledge in different disciplines only to address with theoretically-based problems via a “faceless” window-based environment. With this restriction, instructors need to deliver their course materials exclusively in digital sources. Usually they act as “experts”, despite another recognized problem that is observed, when students cannot recall or take the appropriate feedback from the instructor or other peers to exchange ideas/opinions for an issue, at the same time or in the same “place” that they want using synchronous communication modes. Thence, a lack of proximity is being raised, hampering the interactive and collaborative climate among them as well. This approach is still a target of criticism in STEM education, because (Hew and Cheung 2014; Laboy-Rush 2012; Meyrick 2011; Sahin et al. 2014): (i) the learning materials are not grounded on students’ understandings based on their actions or experiences gained from the real world; (ii) the self-directed learning or problem solving skills are not sufficiently cultivated, because students are not involved in practical hands-on experiments to acquire information or to argue on how effectively the gained knowledge through experiments can be utilized not only efficiently, but also effectively for an educational purpose; and (iii) students cannot develop or enhance their skills to apply the acquired knowledge in realistic problems, which cannot be provided in 2-D LMS.

Among various e-learning platforms that have been used in the past, such as LMS or MOOC, 3-D multi-user virtual worlds (or environments/MUVEs), such as Second Life, Open Sim etc., can provide for STEM education alternative and effective options, which can affect not only students’ motivation, but also their meaningful participation in collaborative tasks based on different socio-cognitive underpinnings, following blended or (fully) online instructional formats (Pellas and Kazanidis 2014). Users’ participation in 3-D multi-user virtual worlds are a powerful magnet for spatially (or not) distributed users, giving them incentives for socialization and collaboration due to the technological and instructional affordances which are offered. According to Dalgarno and Lee (2010), these “affordances” represent the theoretical learning benefits of 3-D multi-user virtual worlds explicitly and/or implicitly purported by the authors in the literature. This term was preferred over ‘benefits’ or ‘advantages’ in favor of referring learning tasks, activities, theoretical underpinnings or pedagogical strategies supported by 3-D multi-user virtual worlds and labeled as “educational potential”.

Previous literature reviews (Dalgarno and Lee 2010; Hew and Cheung 2010) have addressed various learning affordances, such as actual applicability and scalability issues that 3-D multi-user virtual worlds offer. Although these are significant reasons for utilizing this technology in STEM education, another literature review needs to be conducted, so as to present: (a) the learning benefits and specific features (pedagogical, technological or financial) that emerged from 3-D multi-user virtual worlds and may boost communication, collaboration and participation among users; (b) the learning outcomes of having students actively engaged in realistic and situated instructional design methods, which in their majority facilitate effectively STEM courses; (c) the impact of instructional design methods incorporated with theoretical underpinnings of learning theories that previous studies have utilized; and (d) how content design characteristics can influence the students’ achievements, by harnessing the technological characteristics of 3-D multi-user virtual worlds. Each of these areas is noteworthy and should be discussed.

According to the aforementioned research findings, new instructional design methods in STEM education must present a new chance to engage students in practical hands-on experiments on knowledge and learning through realistic conditions and should be addressed using 3-D multi-user virtual worlds. Exploration of instructional design methods based on learning theories can provide a pedagogical framework in order to understand instructional technologists and scholars the educational potential of 3-D multi-user virtual worlds and finally utilize them as learning platforms in different STEM disciplines. The general hypothesis that this review attempts to answer firstly, is if the use of 3-D multi-user virtual worlds in STEM courses can impact students’ engagement and participation positively; and secondly, is if the implementation of different instructional frameworks can facilitate the teaching and learning process, with the intention to produce acceptable learning outcomes.

This study intends to review recently published scientific literature on the use of 3-D multi-user virtual worlds in STEM education, in order to: (a) identify the potential contribution of 3-D multi-user virtual worlds as candidate learning platforms in Primary (aged 7–12 years old at elementary school level), Secondary (aged 13–17 years old at middle/high school level) and Higher education (aged from 18 and up at post-secondary and university level), regardless the instructional format that was followed, i.e. hybrid or online; (b) investigate the theoretical underpinnings based on contemporary learning theories that previous studies followed to construct their instructional design methods; (c) present a synthesis of the available relevant literature to amplify the educational effectiveness or positive learning outcomes achieved by utilizing 3-D multi-user virtual worlds; (d) concretize the educational potential and affordances based on reviewed studies, covering the pedagogical (socio-) cognitive, technological-operational and organizational-financial perspectives; and lastly (e) provide an instructional design workflow to contextualize pedagogical content design principles for STEM courses by using 3-D multi-user virtual worlds.

1.1 Categorization of 3-D multi-user virtual worlds

Bell (2008) defines a 3-D (multi-user) virtual world as “a synchronous, persistent network of people, represented as avatars, facilitated by networked computer” (p. 3). Due to the 3-D visually-rich graphical user interface, users (spatially distributed or not) can interact with a−/synchronous communication channels with others as embodied representations (avatars). A 3-D multi-user virtual world provides interactive simulations in a plausible illusion and it permits users (i) to construct realistic situations using simulated 3-D modelled environment on real places and several constructions with built-in tools and geometric objects; (ii) to refine rules of the spatial proximity with high fidelity; and (iii) to restructure the dynamic dimensions of the knowledge transfer in socio-constructivist learning contexts, in order to improve their achievements (Pellas 2014). The three main categories of 3-D multi-user virtual worlds based on the reviewed studies are the following:

  1. (a)

    Social virtual worlds (SVWs): Using SVWs, thousands of users co-exist in a 3-D persistent environment without specific purposes, progressive storylines or classification levels, on the part of upgrading their virtual entities’ (avatars’) power. The most well-known virtual world is Second Life (SL) with up to 2 million users being connected on a daily basis (Vrellis et al. 2016).

  2. (b)

    Open source virtual worlds (OSVWs): OSVWs have an open-ended technological infrastructure in different server modes (networked or standalone), where users interact and create their own virtual environment (grids). All users are involved with others on the part of co-creating or coordinating their activities, using programming scripting languages “open” to all users without financial cost for constructing a virtual environment. Two of the most well-known open source virtual worlds are Open Simulator (or Open Sim) (Pellas 2016) and Open Wonderland (Ibáñez et al. 2012).

  3. (c)

    Collaborative virtual learning worlds (CVLWs): The educational use of CVLWs has become even more useful for blended or online instructional formats. Distributed users who are separated spatially or temporally, can work as teams to the extent of co-constructing a knowledge domain, by co-existing in a common virtual environment and by interacting through a−/synchronous communication tools. Some indicative examples of this category are Active Worlds (Bouta et al. 2012), Quest Atlantis (Barab et al. 2005), Multiverse (Sancho et al. 2012) and Aeroquest (Okutsu et al. 2013).

Generally, 3-D multi-user virtual worlds can provide alternative and effective options due to the built-in learning tools and a−/synchronous communication tools, fostering not only students’ motivation, but also their meaningful participation in collaborative tasks in blended or (fully) online instructional formats (Pellas 2014). Users’ participation in online 3-D multi-user virtual worlds as a powerful magnet drawing users worldwide, offering incentives for socialization or collaboration to achieve common learning goals. The functionality of a 3-D multi-user virtual world this technology provides some important perspectives. These are the following: (a) the highest response of the system with also the optimal simulations in 3-D realistic conditions and a/synchronous tools to be included (Kim and Ke 2016); (b) the 3-D multi-user environment can affect positively students’ collaboration, responding adequately to their actions in real-time with visual or auditory stimuli. This might guarantee the high representational fidelity in-world activities in which avatars are engaged (De Lucia et al. 2009); and (c) the reinforcement of psychological “immersion” due to the scene of (co-) “presence” (being there with other avatars) in a 3-D virtual environment for a common purpose (Dalgarno and Lee 2010).

1.2 Technological capabilities

3-D multi-user virtual worlds (SVWs, OSVWs CVLWs) can support computer-supported collaborative learning (CSCL) scenarios effectively (Bouta et al. 2012; Konstantinidis et al. 2010; Pellas 2016), affecting positively students’ participation and the development of higher order and cognitive thinking skills. Additionally, educational organizations and institutes have demonstrated considerable interest in exploring teaching and learning possibilities (Bouta et al. 2012; Burgess et al. 2010; Pellas and Kazanidis 2014). Furthermore, 3-D multi-user virtual worlds can engage and attract learners’ attention in various ways based on some technological capabilities that differentiate them from other 2-D digital learning platforms:

  • The sense of (co-)presence that most users can “feel” when they are immersed in a virtual grid allows their co-existence in a common virtual environment to (re-) construct metaphorical representations (metaphors) of their ideas without spatio-temporal physical or digital constraints (Dalgarno and Lee 2010).

  • The different types of communication using verbal (VoIP) calls or non-verbal channels (e.g. gestures or facial expressions that compose each user’s emotional state, IM and chat text). These tools can facilitate the interaction among users in a common virtual environment (Esteves et al. 2011).

  • The embodiment representations of users as cyber entities (avatars) allow the efficiently interaction with their peers in a common 3-D virtual grid. At the same time, they can also use representational functions or artifacts with high fidelity (Okutsu et al. 2013).

  • The expressiveness of animated and interactive 3-D graphical representations of virtual entities (avatars) or virtual places (grids) can be used for the presentation of interactive concepts that are difficult to comprehend in digital or textual forms. With virtual metaphors, users are able to construct meaning within a persistent environment and communicate freely with others to better understand a learning situation in a collaborative climate (Pellas and Kazanidis 2014).

  • The real-time simulation using interactive artifacts are provided for the implementation of different learning scenarios. Students can use the appropriate tools or artifacts for experiential learning and problem solving activities. The creation of 3-D simulations and micro worlds can enhance knowledge representation of the explored domain (Koutsabasis et al. 2012).

Recent studies (Girvan et al. 2013; Mikropoulos and Natsis 2011; Pellas and Kazanidis 2014) have also proposed the functional capabilities provided by the use of 3-D multi-user virtual worlds as “constructivist tools” beneficial for enhancing the learning process in different learning disciplines.

2 Methods

According to Pearson et al. (2014), a mixed method systematic review enables the evaluation and interpretation of all relevant studies, where results extracted from quantitative, qualitative or the synthesis of both research methods (mixed method) are synthesized to answer to one or more relevant research questions, subject matter or even a state of interest. If two or more types of evidence are examined within one review, it is called a “mixed-method” systematic review responded to the appropriateness, feasibility and meaningfulness of previous studies’ data. Basic guidelines by Pearson’s et al. (2014) followed in this review to be described a framework for the synthesis of previous studies results.

A mixed-method systematic review can be useful in the area of ​​3-D multi-user virtual worlds in STEM education for the following reasons: (a) in the systematic effort of setting up the learning material to study, educators and instructors need to propose a new approach or synthesize an innovative (qualitative or quantitative) research efforts emerging from the findings of previous studies. In this study, all reviewed studies present results from the implementation of different instructional design methods as a means to address specific problems, (b) it is needed for the instructional technologists to recognize the capabilities, which are raised by utilizing 3-D multi-user virtual worlds, so as to secure significant funding grants for primary research in STEM education, and (c) many researchers start to explore new ways with a view to practice and empower students’ skills/abilities and present ideas or understand theorems.

Therefore, a synthesis of previous researches in this field and an analysis of major concerns rising from different research methods through a systematic review can assist not only to the establishment of better research material, but also to the investigation of all features that can provide a new impetus for the implementation of innovative instructional methods in STEM courses. By adopting the methodological framework of Pearson et al. (2014), this mixed-method systematic review covers the following stages and activities:

Stage 1: Planning the review.

Activity 1.1: The rationale and the need to conduct this review.

Activity 1.2: Search strategy and development of a protocol.

Stage 2: Conducting the review.

Activity 2.1: Identification of research questions.

Activity 2.2: Selection/exclusion of studies.

Activity 2.3: Study quality assessment.

Activity 2.4: Coding scheme of the reviewed papers.

Activity2.5: Data synthesis (aggregative findings from studies that used qualitative and quantitative methods which being used in each study).

Stage 3: Reporting the review.

Activity 3.1: Communicating and synthesizing the results.

2.1 The rationale and the need to conduct this review (stages 1 and 2)

A sense of obligation to aggregate and present results emanated from this review of fifty (50) articles, in which results from experiential-based and case studies are described. Therefore, it is necessary to describe data from qualitative or quantitative research methods that held in 3-D multi-user virtual worlds and conducted in different STEM fields of Primary, Secondary and Higher education with 8, 9 and 33 studies respectively. These studies were the most relevant for the synthesis of the present literature review (Activity 1.1).

Based on previous literature reviews’ findings, it was observed that all of them focused on studies, which were conducted in order to present results from previous studies, using Second Life or other 3-D collaborative virtual learning worlds. In chronologically order, results from previous literature reviews have presented below:

  • The short description of different learning courses that held in 3-D multi-user virtual worlds has covered only higher education disciplines (Lee 2009).

  • The systematic review of literature in Primary, university and polytechnic settings, in which the use of 3-D multi-user virtual worlds presented (Hew and Cheung 2010).

  • The potential use of Second Life for different “edutainment” (education and entertainment) learning approaches in K-12 and Higher education (Inman et al. 2010).

  • The inherent technological characteristics, pedagogical approaches and students’ perceptions of a review from fifteen (15) case studies which were presented (Dass et al. 2011).

  • A comprehensive literature review focused on virtual learning environments, in which information-related design distinguishing characteristics are derived. However, neither all these environments separated according to their technological infrastructure, nor all are separated based on the educational levels usage (i.e. Primary, Secondary or Higher education) (Mueller and Strohmeier 2011).

  • The conceptual analysis of educational contexts, features, characteristics and learning theories that were mainly followed by instructors who analyzed the results only from educational virtual worlds (Mikropoulos and Natsis 2011). However, 3-D social or open source virtual worlds were not included.

  • The advantages and disadvantages of utilizing 3-D multi-user virtual worlds in education raised from the analysis and synthesis of 100 peer-reviewed articles generally, without standing in specific learning disciplines (Duncan et al. 2012).

  • The descriptive findings of empirical studies based on fifty-eight (58) peer-review Journals. However, all of them have presented cases using Second Life as a learning platform (Wang and Burton 2013).

Although all these reviews provide an overview in various research issues concerning 3-D multi-user virtual worlds that impact the “status quo” of education, they are limited in scope, because of their focus on different aspects. In view of the increasingly complex, dynamic and multidisciplinary nature of STEM education, this study adopted a mixed-method systematic review on this broad area of research to analyze the domain as a whole in terms of its knowledge structure or research themes. General points of view gathered from the above reviews have inductively shown the following issues:

  1. (a)

    The educational potential of 3-D multi-user virtual worlds (open source, social etc.) have not been thoroughly investigated or conducted in previous studies. More specifically, the utilization of 3-D multi-user virtual worlds for different educational levels in STEM education is still lacking.

  2. (b)

    According to this literature, some of the most well-known digital academic sources (e.g. ACM Digital Library, Springer, Taylor and Francis, Wiley) for articles with important contributions to 3-D multi-user virtual worlds in education have not been examined yet. Thence, their investigation seems to be imperative.

  3. (c)

    Second Life was found as the most prominent and well-known social 3-D multi-user virtual world. However, it was not examined from the previous literature reviews the category of 3-D open source virtual worlds or other 3-D collaborative virtual learning worlds that have been utilized in Primary, Secondary or Higher education generally, and in STEM disciplines more specifically. Hence, a study that may inspire educators to systematically start to re-think other 3-D virtual worlds as candidate learning platforms and present the educational potential which can overlap the financial or pedagogical constraints is still needed.

  4. (d)

    the empirical or case studies that exploited 3-D multi-user virtual worlds in Secondary education, like the Schome project in Teen Second Life (Twining 2009) or other studies which have used 3-D open source virtual worlds (Pellas 2016; Rico et al. 2011) are not sufficiently presented in previous literature reviews.

Two are the significant observations that motivated this review conduct. Firstly, Mikropoulos and Natsis (2011) have referred that very few studies had a clear theoretical (pedagogical) model to inform the use and design of the 3-D virtual worlds for educational purposes. “All the other reviewed articles do not refer explicitly to a learning theory,” (p. 775) as the same authors have noted. According to the same authors’ statement, three needs that should be highlighted in a new review are the following: a) the need of identifying new learning requirements from elicitation and specification methods that can be used in 3-D multi-user virtual worlds, b) the need of specifying rigorously new instructional design guidelines in a coherent instructional design workflow to construct valuable and viable learning environments that can be held in 3-D multi-user virtual worlds, and lastly c) the need for innovative instructional formats guide to the use of this technology more effectively in everyday teaching and learning of STEM courses.

Secondly, Dalgarno and Lee (2010) have stressed of great importance in establishing guidelines and best practice, in order to amplify towards the educational benefits and affordances of using 3-D virtual words for educational purposes. Practical hands-on experiments should include an instructional design framework that include basic principles of organizing and managing successfully students’ experiences during the learning process in STEM courses. Thence, it is needed a study to get focused on eliciting the learning requirements from pedagogical models and instructional design frameworks that previous studies implemented in their learning scenarios, and then seek to create a strong evidence of the technological support.

At this time, the analysis and synthesis from the results of the relevant literature in a mixed method systematic review should be in association with: (i) the instructional design methods based on theoretical underpinnings of learning theories that users (instructors and students) have implemented using 3-D multi-user virtual worlds, (ii) the results of learning foci (i.e. for the identification of previous studies’ purposes, hypotheses or questions) according to a summary of significant findings and implications, (iii) the instructional formats, research methods, content design or implementations of practical hands-on experiments that held in 3-D multi-user virtual worlds, and finally (iv) the educational potential and benefits (affordances) of this technology in STEM education.

Summing up all the above, the main intention of this study is firstly to review extensively and systematically the educational potential of 3-D multi-user virtual worlds rising from the relevant to STEM education studies, and secondly to contribute toward an analysis of learning and research methodologies that being used. The problematic assumption that this study seeks to explore is the lack of a literature review that should present and analyze extensively the educational potential and affordances of 3-D multi-user virtual worlds which can foster students’ engagement and affect positively the learning achievements in different STEM education levels. In these dimensions, a review of the recent literature is conducted to overcome some limitations of previous reviews which should be seriously take into account. Lastly, an instructional design workflow that aimed at attempting a more pedagogical description using the concept of pedagogical and functional perspectives for mapping stages of learning into types of learning environment and extending their road map for further research in STEM area is also proposed. This workflow has “design for learning” perspectives and in doing so it would be useful to those instructional designers and educators who design learning activities in 3-D multi-user worlds.

2.2 Search strategy and the development of a protocol

The following resources were searched to identify and collect relevant material. The searched databases and included articles that identified as relevant to STEM education were from: ACM Digital Library, ScienceDirect, ERIC, SpringerLink, Wiley Online Library, Taylor & Francis, MIT press. Web searches were conducted using the Google Scholar as search engine. Branching searches were performed using forward and backward search procedures from the reference lists of some previous literature reviews that were located in earlier stages of this review. Table 6 in Appendix shows the development protocol that was executed for each database (Activity1.2).

2.3 Identification of research questions

Initially, a research to identify the existence of systematic reviews, involving 3-D multi-user virtual worlds in STEM education was followed. Specific research on this subject was not found. However, there were several literature reviews involving the use of this technology in education, as the rationale of this study underlines (see subsection 2.1). Within the context of this paper, a mixed method systematic literature review was carried out, using the basic review approach that Pearson et al. (2014) have proposed, in the interest of examining the state of studies in educational potential of 3-D multi-user virtual worlds. Therefore, the present study is based on the following research questions (RQ) (Activity 2.1.):

RQ1: What topics are taught through the utilization of 3-D multi-user virtual worlds in different STEM disciplines of Primary, Secondary and Higher education?

RQ2: Which instructional design methods (or learning approaches) based on contemporary theoretical underpinnings have taken into previous studies under consideration in STEM disciplines using 3-D multi-user virtual worlds?

RQ3: What are the educational potential and affordances of 3-D multi-user virtual worlds that can contribute to the effect of students’ engagement and positively impact their learning outcomes?

RQ4: Can an instructional design workflow be identified according to the reviewed papers with a purpose of contextualizing the pedagogical and technical instructional design principles by utilizing 3-D multi-user virtual worlds in STEM courses?

2.4 Selection and exclusion criteria

Disciplines which are included in this review were from topics in the fields of STEM Education and Learning Research generally, and specifically from Computer science, Chemistry, Physics, Information Technology Science, Programming, Science Technologies to Engineering, Mathematical and Physical sciences (see recommendations from Chen and Thomas 2009). Using these disciplines, 50 papers met the inclusion criteria of this study (Activity 2.2).

These entries were narrowed down further by focusing on papers, which: (a) included an empirical or a case study evidence relating to instructional design method that have been used, eligibility, applicability and affordances in 3-D multi-user virtual worlds; (b) discussed students’ engagement in specific topics of STEM education; (c) focused on students’ positive or negative learning aspects in their outcomes or achievements; (d) dated from January 2000 until the first quarter of 2016; and (e) published in peer-reviewed and refereed academic journals written in English.

The following five criteria were used to determine which of these papers that should be included in this review (due to their consistency with the educational context that analyzed):

  1. 1.

    Articles should report the utilization of a 3-D multi-user virtual world in a specific STEM discipline and in a specific educational level.

  2. 2.

    Articles included in this review only if a quantitative or qualitative (or mixed research methods) research evaluation method was presented.

  3. 3.

    Articles should implement learning scenarios in 3-D multi-user virtual worlds, taking under consideration the theoretical underpinnings of contemporary learning theories or models or/and instructional design principles for pedagogical practical hands-on experiments analyzing STEM educational-theoretical concepts or theorems.

  4. 4.

    Studies that measured the learning gain, i.e. learning outcome variable using test instruments, observation of students’ performances, and students’ work samples.

  5. 5.

    Studies should use experimental research design to measure relationships between in-world instructions and learning experience.

In order to study the educational potential of the reviewed studies, the following categories should be identified: (a) in-world instructional design methods or models based on theoretical foundations, and the research methodologies that measured the successful implementation of STEM courses in 3-D multi-user virtual worlds, (b) the purpose of these studies (the scientific construction and understanding of knowledge), (c) the students’ engagement in different PBL or PrBL approaches, the effectiveness of instructional design methods in 3-D multi-user virtual worlds, (d) the socio-cognitive development of leaning gain in STEM disciplines, (e) the students’ engagement and the impact on their learning outcomes.

In this review, books, book chapters, symposia, project reports and workshops were excluded. Also, all articles that did not present data from evaluations or were not organized in well-structured research methods (case studies, empirical etc.) or were not written in English and were before 2000 or after the first quarter of 2016 were not included. Furthermore, neither augmented (mixed) reality environments nor virtual classrooms have been included because those “mixed realities” were different from the virtual environments by combining virtual with real situations in face-to-face settings.

The following five criteria were used to determine which papers should not be included in this review (due to their consistency with the educational context that analyzed):

  1. 1.

    Articles that aimed at teaching different topics than STEM disciplines by utilizing of 3-D multi-user virtual worlds.

  2. 2.

    Articles that did not entail a quantitative or qualitative assessment of learning. If an article did not present interviews, researcher’s observations (qualitative research method), and it also was not given a specific hypothesis or statistical analysis of users’ answers (quantitative research method).

  3. 3.

    Articles that did not show the use of 3-D multi-user virtual worlds in a STEM course, involving automated equipment or 3-D simulation-based environments. Studies that used desktop-based virtual reality technologies (e.g. Cave Automatic Virtual Environments) as an assessment, diagnostic, therapeutic platform or learning scenarios for the disciplines of STEM education were not relevant for this review.

  4. 4.

    Articles that considered out of context of addressing in different STEM fields of primary, middle, high school, post-secondary and higher level of education. Some studies that reported out of the design of 3-D multi-user virtual worlds or the familiarity of pre-service and instructors with 3-D multi-user virtual worlds among other aspects were not considered in this review.

  5. 5.

    Studies that did not provide sufficient data for effect size calculation or studies that did not have clear summarization or aggregative findings from their qualitative data.

Different 3-D multi-user virtual worlds and research methods (case or empirical studies) are presented in this present review. According to the Thomson Reuters Journal Citation Reports (2012–2013) almost all journals that accepted these reviewed articles, have impact factors between 0.532 and 2.676. This distinction indicates that this technology is a relatively new field for researchers and educators who want to utilize 3-D multi-user virtual worlds as an up to date solution for the implementation of practical hands-on experiments in STEM courses.

2.5 Main literature search

From the overall ninety-four (94) papers which were published in different peer-reviewed Journals, only 50 identified as appropriate, according to the inclusion criteria of this study: (a) 12 papers were observed in Primary education, however only 8 of them were the most appropriate, (b) 20 papers were identified for the utilization of 3-D multi-user virtual worlds in Secondary education, with 9 of them to be pertinent for this study, and finally (c) the vast majority of papers have implemented their learning scenarios in Higher education (62), with only 33 of them to be focused in STEM education.

Also quantitative, qualitative and mixed research design methods were observed at all different STEM education levels using 3-D multi-user virtual worlds. Firstly, for the Primary education, 4 quantitative, 2 qualitative and 2 mixed methods were observed. Secondly, for the Secondary education, 5 quantitative and 4 qualitative methods were followed. Lastly, for the Higher education, 15 studies had followed quantitative method, 13 qualitative and only 5 have followed mixed method research approaches.

2.6 Study quality assessment

The selected papers were separated in two categories (Activity 2.3). The selection of the reviewed papers which used the quantitative analysis of results considered as the most accurate form of experimental research to prove or disprove a hypothesis through statistical analysis. For an experiment to be classified as a true experimental design, the following criteria must fulfil (Russell and Gregory 2003): (a) the research question was clearly defined and adequately substantiated, (b) the method of sampling should be appropriate for the research questions and instructional design methods, (c) the data analyzed appropriately, (d) the analytical description of findings, (e) the meaning or relevance of the study need to have some practical implications for knowledge acquisition.

The selection of the reviewed papers for qualitative analysis, which were opted as the most accurate form based on the researcher’s observations or sample’s interviews to prove or disprove research questions, could be answered through: (a) a purposeful sampling based on a conscious selection of a small number of data sources, (b) the logic and power of purposeful sampling in selecting information rich cases (participants or settings) for in-depth study to illuminate the questions of interest, covering a wide range of potentially relevant social phenomena and perspectives from the utilization of different instruments, and (c) the descriptive analysis of a course, trying to explore new cases or perspectives that may lead in future-driven exploitation of 3-D multi-user virtual worlds.

2.7 Coding scheme of the reviewed papers

A coding scheme with the main criteria for each article is provided. These are as follows: (a) the instructional design method that was implemented, (b) the selection and promotion of the most unique technological characteristics exploited in STEM education, (c) the theoretical underpinnings, based on contemporary learning theories that followed, (d) the instructional design using 3-D multi-user virtual worlds to enrich visually-rich learning settings that impact students’ achievements and positive learning outcomes.

Below, a coding scheme of the data extraction in documenting their inclusion for this literature review is presented (Activity 2.4).

figure a

The first author of this study conducted all of the content analysis results and finally the data was validated by the other three authors who are experienced researchers in Informatics and Science education. The other three authors who have individual researchers with basic or master degrees or professorship in the educational technology domains conducted the screening tasks and they need to discuss with the first author in the direction of reaching the overall consensus based on the described selection rules. All authors needed to read the full text of each article in order to decide if it could be included in this review (Activity 2.5).

2.8 Data collection and content analysis tool

The present article reviews empirical or case studies, in which different research methodologies (quantitative, qualitative or both using a mixed research method) were followed to be measured the impact of 3-D multi-user virtual worlds on learning outcomes and published in full length peer-reviewed Journals. For the best processing of the study analysis and reliability results, Nvivo (ver. 10) software was used in an effort to perform a content analysis of the reviewed studies analytically.

3 Results (stage 3)

3.1 Data analysis

In this review, as the samples, instructional design methods, research and data collection methods differed so greatly, it was not possible to undertake a meta-analysis. The overall findings were synthesized to extract the main themes under which the findings of the review are identified and presented. The process was inductive; therefore, there were no a priori themes assigned to the data. Each paper was read several times, and codes assigned to individual findings were recorded in the tables below.

Firstly, for the RQ1 (Activity 3.1). General points of view from Primary, Secondary and Higher education studies are analyzed alphabetically. These are: (i) types of 3-D multi-user virtual worlds, (ii) learning approaches, (iii) theoretical foundations, (iv) learning foci, and finally (v) research methods and tools for the analysis of results (Tables 1, 2 and 3).

Table 1 General overview of Primary education studies
Full size table
Table 2 General overview of Secondary education studies
Full size table
Table 3 General overview of Higher education studies
Full size table

The research purposes in Primary, Secondary and Higher education utilizing 3-D multi-user virtual worlds are depicted briefly in Table 4. Secondly, all the selected articles from various perspectives, were examined: (i) the comparison of learning effectiveness in face-to-face or virtual class settings, (ii) the utilization of innovative learning methods, (iii) effectiveness and usability of 3-D virtual worlds in different STEM disciplines, (iv) the content design and instructional affordances, (v) investigation of students’ engagement.

Table 4 The research purposes from Primary to Higher education via the utilization of 3-D multi-user virtual worlds
Full size table

3.2 Categorization of papers

To fully answer RQ2, two aspects need to consider: (i) in what educational contexts and under which instructional design methods were the courses conducted successfully; (ii) which were the learning cores and learning foci of the reviewed studies (types of 3-D virtual worlds that used, students’ engagement with the learning material and educational context), and (iii) which was the in-world instructional design of the experiment (see subsection 3.2 and 3.3 of this review).

The analysis of the literature was based on the four levels of theoretical underpinnings including theory, model, approaches, and learning principles. The learning cores (learning foci) of relevant studies were mainly identified through the research purposes, questions, or hypotheses and sometimes through research instruments or results when it was necessary. The scrutiny of this literature review was useful to organize papers into categories, depending on the main focus of each paper. The categories are shown in Table 5 along with the number of papers in each of these categories and were derived from consideration of different stages in various learning processes. Learning cores, studies descriptions, in-world instructional design, and studies of Primary, Secondary or Higher education based on STEM education are also included according to the analysis of Table 5.

Table 5 Categorization and description of learning focus
Full size table

It can be concluded (see results provided in Tables 1, 2, 3, 4 and 5) that mainly: (a) DGBL settings for students of Primary education were used; (b) simulation-based activities for students of Secondary education indicated as the most appropriate for their participation; and (c) different socio-cognitive instructional design frameworks for various practical hands-on experiments in Higher education according to students’ needs and demands were the most appropriate for teaching and learning innovation.

The most significant aspect was that all authors who have conducted previous studies had a positive impact on learning, regardless of the students’ cognitive background and their information literacy skills. Students’ participation was not only for game-playing reasons, but for the best understandings of theoretical concepts or theorems in STEM disciplines using different learning approaches.

3.3 Analysis of the reviewed studies

A need for appropriate instructional design methods using 3-D multi-user virtual worlds in practical hands-on experiments for STEM courses is widely recognized. The utilization of 3-D multi-user virtual worlds in STEM education has focused on different socio-cognitive theoretical underpinnings (Bouta et al. 2012; Pellas 2016). Socio-constructivist learning approaches have been successfully applied in 3-D multi-user virtual worlds due to a number of unique affordances that can sufficiently support purposeful collaborative and meaningful practical hands-on experiments (Bouta et al. 2012; Dickey 2003; Ketelhut 2007; Girvan et al. 2013).

In Primary education, several studies (Barab et al. 2005; Bouta et al. 2012) have utilized 3-D multi-user virtual worlds in different STEM disciplines. Hickey et al. (2009) have found that sixth grade students who received the science courses (Ecology) in 3-D multi-user virtual worlds demonstrated greater learning gains in understanding the scientific concepts and in achieving information or learning of materials, in contrast to those of a control group (using expository text) who did not perform so well. The same authors also reported that formative feedback in digital game-based settings improved students’ achievements. In-class and online settings have positively encouraged students to participate in pair collaboration between students in order to enhance their socio-cognitive skills. Choi and Baek (2011) have shown that multimedia characteristics of 3-D multi-user virtual worlds, like ‘interactivity’, ‘representational fidelity’, ‘immediacy of communication’, ‘consistency’, and ‘persistence’ have influenced students’ engagement and therefore are provided as useful constructs for the instructional design content. Moreover, Lim et al. (2006) have followed the learning principles of Constructivism to establish students’ current level of understanding in Science topics, especially on Water Cycle, Water Purification, and Water Pollution using Quest Atlantis. Students’ engagement and understanding of these phenomena increased. Quest Atlantis seemed to allow elementary students, specifically in after-school programs, participate in this virtual environment. Results showed that participants performed better in educational activities, communicate and collaborate with other peers or instructors, and build their own constructs, following a socially-responsive instructional design method. The same authors have mentioned that students started the creation of their own learning environment in the interest of engaging in Science courses. In Barab et al. (2012), students enhanced their cognitive development using a DGBL approach. However, students’ difficulties with the language used in 3-D multi-user virtual worlds and a lack of computer competency caused their dissatisfaction (Barab et al. 2012). In Mathematics, students were taught in collaborative instructional settings using a macro-script to design in-world skills of translating the concept from one system of representation into another (i.e. to complete a symbolic, graphic or verbal representation of a fractional concept). The purpose was to transfer their tasks into a 3-D multi-user virtual world (Bouta et al. 2012). The analysis of an instructional design framework based on the Jigsaw teaching technique and combined with a macro-script, helped students to understand the design principles in collaborative settings and thus to better comprehend the instructor’s role in this environment.

In summary, one study was performed in Active Worlds (Bouta et al. 2012), and five in Quest Atlantis (Barab et al. 2005; Barab et al. 2012; Hickey et al. 2009; Lim et al. 2006; Yong and Ping 2008). All the above studies used 3-D multi-user virtual worlds as media or candidate learning platforms and collaborative tasks implemented in the interest of discovering or understanding scientific knowledge concepts among users to solve more complex problems. An important observation is that neither social nor open source virtual worlds were used in Primary education.

Regarding Secondary education, educators and scholars have created several effective learning environments in 3-D multi-user virtual worlds, which were more easily adoptable to the students’ needs and demands. Accordingly, they were able to understand the opportunities which these environments can offer, depending on the pedagogical approach and theoretical background that met their teaching and learning needs. 3-D multi-user virtual worlds began to be utilized after 2002 to fulfill educational goals and increase students’ participation. Important aspects of designing learning activities usually are focused on the theoretical knowledge underpinned by Situated Cognition (Barab et al. 2005; Dickey 2005; Ketelhut 2007; Nelson and Ketelhut 2007), Constructivism (Corbit 2002; Omale et al. 2009; Xu et al. 2011) or Socio-constructivism (Twining 2009). As for the design of the research conducted in several studies of Secondary education, more emphasis was given on simulation, experimentation and exploration of concepts covered in the field of Science education (Corbit 2002; Nelson and Ketelhut 2007).

Dickey (2003) has mentioned that built-in learning tools, such as whiteboards or resource tools in 3-D multi-user virtual worlds can assist the learning process. However, these tools are missing, mainly from these “worlds” because are not for educational purposes. This of course is understandable, because social or open source virtual worlds are not learning platforms per se. Nevertheless, such platforms are still considered to be candidates due to their technical infrastructure having capabilities that can be adapted and configured from all users, according to their needs or demands. Second Life can also provide built-in tools for the configuration of a learning environment, although the tools made learning practice easier to manage, the maintenance cost is still quite high. Another study referring to Information and ICT courses suggests that “lived experiences” in High schools via Teen Second Life can assist students’ understandings in practical realistic problems, as this process enhanced their engagement (Twining 2009). Ideally, these perspectives highlighted instructional affordances for the design content with characteristics like cooperation, socialization, retention, and attendance rates at high levels, empowering students’ experiences.

Using 3-D multi-user virtual worlds, students are engaged in meaningful quests through game-based or story-based settings (Barab et al. 2005). Several cases (Xu et al. 2011; Young et al. 2012) have implemented game-based learning activities by utilizing built-in tools and the a−/synchronous tools of Second Life. This “world” as a curriculum-aid platform was utilized in a National Science Foundation (NSF) project known as STEAM (Science and Technology Enrichment for Appalachian Middle scholars), through the design, creation, and deployment of three educational games, as proposed in the Young et al. (2012) study. Worthwhile are also the requirements of design principles placed on collaborative activities that students need to utilize in the interest of participating for knowledge acquisition in Computer Science courses (Rico et al. 2011). The results were very positive as pre-test and post-tests indicated that students learn collaboratively how to think and exchange their ideas with their peers, before executing a code. The results of linear correlations to amplify the interrelationships among presence indicators (cognitive, social, and teaching) of a CoI model to learn basic programming concepts via a 3-D multi-user game-like environment underpinned by Papert‘s theory of Constructionism was studied (Pellas 2016). The findings indicated that social presence (communication and cohesiveness of a group) had not only a direct correlation with the cognitive presence (learning process for the construction of knowledge), but also had a positive association with teaching presence (organization, planning, and guidance of learning activities), reinforcing them as well.

Two studies were performed in Active Worlds (Corbit 2002; Dickey 2003), one in River City (Ketelhut 2007), two in Second Life (Omale et al. 2009; Young et al. 2012), one in Open Sim (Pellas 2016) and another one in Teen Second Life (Twining 2009) and in other one case, the authors created 3-D world-prototypes (Xu et al. 2011). All studies which utilized 3-D multi-user virtual worlds in collaborative tasks for STEM education have as their main scope to improve student engagement and the acquisition or enhancement of their skills (i.e. higher-order, problem-solving collaboration or communication skills).

In Higher education, Second Life was a dominant platform for the implementation of different learning activities in STEM education (Beltrán Sierra et al. 2012; Vahey et al. 2011; Vrellis et al. 2016). Few studies were focused on the simulation of realistic objects through which learning can be achieved (Koutsabasis et al. 2012; Okutsu et al. 2013; Vosinakis and Koutsabasis 2012), while others have preferred to implement their scenarios using several visually rich representations for the modeling of cognitive structures (Ibáñez et al. 2012; Merchant et al. 2013; Vosinakis and Koutsabasis 2012). Several studies (Koutsabasis et al. 2012; Okutsu et al. 2013; Vahey et al. 2011; Tüzün 2007) have presented the advantages in using 3-D visual-spatial representations as the most significant features of real life metaphorical representations. Moreover, instructional design principles in constructivist-oriented instruction are presented in some studies as a way of representing a thought using cognitive objects (Koutsabasis et al. 2012; Okutsu et al. 2013; Pellas 2014; Vosinakis and Koutsabasis 2012).

A variety of studies proposed specific instructional design methods and models that were applicable to different activities in Higher. Apart from the above, de Freitas et al. (2010) have proposed a four-dimensional framework (i.e. Learner specifics, Pedagogy, Representation and Context) for designing and evaluating learning experiences in Second Life. Particularly, Bronack et al. (2006) have presented a socio-constructivist instructional design method for online educational activities that encouraged cross-class collaboration for students’ interaction. Girvan and Savage (2010) have applied Communal Constructivism as a potential instructional design theory for the pedagogical use of Second Life, as they believe that this foundation can be valuable for the implementation of technology supported courses. Other studies substantiate that Second Life is appropriate for collaborative project-based (Esteves et al. 2011; Girvan et al. 2013; Jarmon et al. 2009) or problem-based learning settings (Jamaludin et al. 2009).

Lorenzo et al. (2012) have found evidence that Massively Multi-user Online Learning (MMOL) systems can lead to more connections among students than LMS platforms by studying the effectiveness of multi-user immersive environments for collaborative evaluation tasks in realXtend, a free open project that extends the feature set of Open Sim. Okutsu et al. (2013) have utilized Quest 3-D Game Engine for developing AeroQuest. The purpose was to design and prototype an aerospace engineering design course in realistic settings. Students had the ability to understand in practice the consequences of their activities in real time. The same authors stressed that lectures need to be delivered in a traditional way with 2-D slideshows, but practical hands-on experiments should be conducted in a 3-D collaborative environment. The low cost of performance was also a crucial factor affecting the exploitation of an open source virtual world in engineering courses. Koutsabasis et al. (2012) have implemented conceptual design constructs in Open Sim and 3-D visual prototyping formats during the completion of complex structures. Beyond the learning goal of students collaboratively designing complex cognitive constructs in the 3-D multi-user environment of Open Sim, their communication, awareness and engagement in practical hands-on experiments were also strengthened. Vosinakis and Koutsabasis (2012) have suggested that 3-D multi-user open source virtual worlds can support constructionist instructional design methods and problem-based learning approaches in HCI (Human-Computer Interaction) design engineering courses. They emphasized on the constructivist-oriented (learning by doing) approach, enabling students to work collaboratively in realistic situations, follow self-directed routes to address problems, and construct digital artifacts as proposed solutions to create digital models that reflect their understanding about their learning. In another study, Koutsabasis and Vosinakis (2012) have proposed that 3-D multi-user virtual worlds aim to aid students’ reflections on the use of related methods and technologies for prototyping and modeling Human and Computer Interaction (HCI) design engineering courses and to cultivate more generally students’ self-directed learning, intrinsic motivation and critical thinking skills.

The utilization of 3-D multi-user virtual worlds has also made a significant contribution to the learning effectiveness in Technology, Informatics and Computer Science courses. 3-D multi-user virtual worlds have been successfully used to implement scenarios following socio-constructivist learning approaches. This technology embraces a number of unique instructional affordances which can sufficiently support purposeful practice-based collaborative and meaningful activities in STEM education. Based on the above, today’s educational uses take advantage of design-based, role-playing or resource sharing capabilities using Jigsaw and Fishbowl teaching techniques. Instructional designers and educators can use them as content design characteristics for educational purposes with the view of reinforcing collaboration among all users (instructor and students) in visually-rich learning spaces (Konstantinidis et al. 2010). Similarly noteworthy, are the results from those educators who have already implemented different Computer science courses for the following reasons: (i) to understand students introductory programming courses via Scratch4SL (a free plug-in of Scratch) and Second Life for experiential game-based learning situations by sharing artifacts with behavior (Girvan et al. 2013), (ii) to learn introductory computer programming using Second Life programming language with regard to replace traditional languages or create “objects-to-think-with” via Linden Scripting language (LSL) in collaborative instructional settings (Esteves et al. 2011); (iii) to learn how to secure virtual grids based on Communal Constructionism (Girvan and Savage 2010); and last but not least (iv) to measure the effectiveness of 3-D game-based settings in an exploratory way through 3-D multi-user virtual worlds (Sancho et al. 2012). Using 3-D multi-user virtual worlds for game-based settings can become a candidate platform for transforming a virtual grid to an incubator of knowledge for students with different cognitive backgrounds who really want to participate in 3-D multi-user serious games even at a distance for supplementary educational activities.

Taking one step further, Pellas and Kazanidis (2014) have suggested that before the students enter into 3-D multi-user virtual worlds, it is always important from the instructor the use of an instructional design framework that must be followed by all students so that eliminate the obstacles observed during the execution of different tasks and acquisition of management responsibilities that students must accept in collaborative climate to achieve common goals. The implementation of an organizational framework according to the presence indicators of a community of inquiry model (CoI) and underpinned by Constructionism proved that students’ self-efficacy and situational interest enhanced their social and cognitive presences respectively. Also, the same authors have mentioned that online and blended university-level courses cannot yet replace traditional teaching methods, but should be used supplementary. The online instructional format in Second Life seemed to be more appropriate for students who want to participate in other future-driven tasks, and students had more positive learning outcomes depending on the degree of their engagement (cognitive, behavioral and social).

Seven studies were performed in the Open Simulator virtual platform (Griol et al. 2012; Kim and Ke 2016; Koutsabasis et al. 2012; Koutsabasis and Vosinakis 2012; Pellas 2014; Prendinger et al. 2012; Vosinakis and Koutsabasis 2012), one in Quest Atlantis (Tüzün 2007), two in Open Wonderland (Ibáñez et al. 2012), one in SimLambia (Erlandson et al. 2010), one in River City (Choi and Baek 2011), one in Active Worlds (Bronack et al. 2006), nineteen in Second Life (August et al. 2016; Beltrán Sierra et al. 2012; Burgess et al. 2010; de Freitas et al. 2010; De Lucia et al. 2009; de Noyelles and Seo 2012; Esteves et al. 2011; Girvan et al. 2013; Jamaludin et al. 2009; Jarmon et al. 2009; Keskitalo et al. 2011; Konstantinidis et al. 2010; Merchant et al. 2013; Pellas and Kazanidis 2014; Vahey et al. 2011; Vrellis et al. 2016), and lastly two studies have utilized their own prototypes for collaborative formats in Multiverse (Sancho et al. 2012) and Aeroquest (Okutsu et al. 2013).

All the above observations are crucial for understanding the role of 3-D multi-user virtual worlds in STEM education. Nevertheless, it is also important to synthesize the results of the reviewed articles in order to acknowledge the instructional and technological affordances of this technology. The proposed framework below provides a synthesis of constructs from which educators and instructors can structure courses in-world, following fully online or blended instructional format. This framework seeks to present and support engaging instructional content design characteristics based on the pedagogical socio-cognitive, technological-operational and financial affordances that 3-D multi-user virtual worlds have offered through unique perspectives in STEM learning design. Using this framework, educators and scholars of STEM education can be influenced to construct engaging, experience-driven lessons that capitalize upon the instructional and technological affordances of 3-D multi-user virtual worlds. Hence, the educational potential and affordances to answer RQ3 are depicted in the following Fig. 1.

Fig. 1
figure 1

Educational potential and instructional affordances using 3-D multi-user virtual worlds in STEM education

Full size image

The advancement of 3-D multi-user virtual worlds seemed that offers the implementation of visually-rich activities, which are capable of supporting a variety of PBL, PrBL, IBL or DGBL learning approaches combined with Constructivist or socio-constructivist theoretical underpinnings. Last but not least, using the learning content in different instructional design methods has generally enhanced students’ engagement for STEM courses. Students in these dimensions have gained a positive attitude toward educational courses and strengthened their positive outcomes in collaborative learning situations. The implementation of learning activities in 3-D multi-user virtual worlds revealed the following educational potential:

  • The design and construction of simulation-based concepts in realistic settings illustrating for users any kind of experiment in CSCL approaches at any scale of space and time without spatial or time limits.

  • The conduct of experiments that can be repeated in 3-D visually-rich (landscaped) environments, without risk or reasonable financial cost, avoiding dangerous consequences of users’ actions.

  • Because there is no risk of damaging the equipment by erroneous handlings, students are not required to follow passive instructions, but they are free to implement active experiments and thus co-construct their knowledge field for CSCL.

  • The “steep learning curve” created during students’ first time entrance in 3-D multi-user virtual worlds can be lowered using free 2-D plug-in modules (see Sloodle or Scratch4SL). Also, these tools can support face-to-face (laboratory) or fully online activities, thus can facilitate the understanding of innovative (learning) phenomena or theorems of STEM education without space or time constraints.

  • The implementation of different learning scenarios based on (socio-) cognitive theoretical underpinnings, following different instructional formats (blended or online).

  • The minimal or free-of-charge financial cost for the implementation of practice-based and problem-based learning scenarios using social, open source and collaborative 3-D multi-user virtual worlds.

Besides the above potential, considerable technological-operational drawbacks which have been revealed are the following:

  1. (a)

    3-D multi-user virtual worlds have high demands on computer hardware requirements, particularly the graphics subsystem and RAM memory. These requirements apply only when they are compared with other digital learning environments (e.g. LMS) which in some cases are more cost-effective. Also, a server as the most important requirement is always required for the creation of an open source 3-D multi-user virtual world, even in personal computers with low technological characteristics (Duncan et al. 2012).

  2. (b)

    the difficulty to obtain participants in educational activities during their first-time entry creates a “steep learning curve”, because of the features which are offered (exploration maps, objects, multimedia content, chat with other users). This problem can be easily solved by the careful selection of content in which learners can take part in each learning activity (Girvan et al. 2013).

4 Discussion

Undoubtedly, throughout this study, the need to construct a strong statement on how and why students of STEM education use and participate in practical hands-on experiments that can be held in 3-D multi-user virtual worlds has emerged. According to the review analysis regarding Primary and Secondary education, the theoretical underpinnings of (socio-) constructivism or (Communal) Constructionism carried out widely. The theoretical foundations of Constructionism seemed to be used in research studies for the conceptual understanding of scientific ideas (Science and Engineering), while in the fields of Technology and Mathematics the socio-cognitive learning foundations appeared as the most eligible and robust. Apart from the implementation of different learning design theories, many articles (Girvan et al. 2013; Koutsabasis et al. 2012) did not provide evidence on how these foundations affected development or knowledge acquisition. To better integrate the learning consequences of using 3-D multi-user virtual worlds, educators and scholars want to invest more efforts for the implementation of an instructional design method based on constructivist-oriented foundations that can be integrated with specific pedagogical principles in order to evaluate the effectiveness of activities.

Students’ participation in 3-D multi-user virtual worlds seemed to be the most significant point of view in (socio-) constructivist instructional design frameworks contributing to this notion on educators and scholars’ decision on making a successful process in STEM education. These findings are also consistent with the results of Lee et al. (2011), who have proposed that motivation factors, and particularly those of engagement, should be further qualified. Students who involved in practice-based learning activities, had free expression of ideas with others, considerably strengthening collaboration among students. Despite the growing interest of instructors and scholars without reasonable technical background for the utilization of 3-D multi-user virtual worlds, they still face significant challenges, if they want to implement different instructional design methods in online or blended settings. Besides the obvious limitations of budget in this financial crisis era and lack of experiences with the new technological improvements, most educators and scholars seemed to not have an innovative thought about what they really want to design with the intention of bringing students closer to the learning material. After thirteen years, 3-D multi-user virtual worlds offer a consideration of why candidate learning platforms remain compelling to educators of STEM education, but also the ways in which they can utilize or capitalize extensively their affordances.

Considering the financial cost of creating content design and implementation of different STEM courses in innovative learning platforms are really crucial parameters for knowledge acquisition, a majority of reviewed literature (Bouta et al. 2012; Dalgarno and Lee 2010; Hew and Cheung 2010; Pellas and Kazanidis 2014; Vosinakis and Koutsabasis 2012) have started to extensively conduct learning scenarios in 3-D multi-user virtual environments for schools, universities and colleges, either private or urban. This technology is mature and appropriate for various pedagogical use, especially for practice-based learning tasks, as STEM courses and programs demand. The unique technological characteristics of 3-D multi-user virtual worlds can be addressed with the multisensory interactive communication channels (a/−synchronous), the built-in tools for the configuration/manipulation of learning grids, the intuitive interactivity through the creation of visually-rich spatio-temporal representations and psychological immersion of users’ “co-presence” in a common and persistent 3-D environment. In this line, other reviewed studies (de Freitas et al. 2010; Konstantinidis et al. 2010; Girvan et al. 2013) have pronounced 3-D virtual worlds as the most appropriate platforms for the reinforcement of a collaborative climate among users, in which they exchange at the same time and in the same place their opinions or ideas propose properly solutions to a problem adequately and provided the appropriate feedback. The exploitation of this technology should be considered as important, in contrary to other 3-D virtual reality systems that are still expensive and awkward for using them in mainstream STEM education.

Dalgarno and Lee (2010) have mentioned that 3-D multi-user virtual worlds can engage students in learning tasks due to the high representational fidelity. This distinction assists the experimentation of different learning contents in STEM education utilizing online or blended instructional formats. These best practices combined with learning theories or models and instructional design frameworks, can provide a solid theoretical and practical foundation with realistic hand-on experience. Instructional designers, scholars and educators should consider the utilization of the inherent technological infrastructure of 3-D multi-user virtual worlds to be vitally important, if they want to deal insightfully in STEM courses using different learning approaches. Some of the most important benefits that must be referred are the following (Prendinger et al. 2012; Sancho-Thomas et al. 2009; Vahey et al. 2011):

  1. a.

    Students can co-construct, co-manipulate and examine in collaborative settings in-world metaphorical representations, artifacts or primitives to design an innovative knowledge domain using (socio-) cognitive theoretical underpinnings. They acquire also the ability to access and experiment in simulation-based learning tasks, without having significant information or technological literacy background.

  2. b.

    A/−synchronous communication forms can allow students to study in a persistent 3-D virtual environment collaboratively. These features engage students more easily, in place of creating diverse teams in small or large groups to study with their peers.

  3. c.

    Learning content design standards for a wide variety of STEM disciplines are more realistic, encouraging relevant standards through practical implications, by following inquiry-based or problem-based learning approaches. The consequences of those approaches were visible for all users who participated and communicate using a-synchronous channels in the same virtual place.

From this point of view, the educational potential and affordances of using 3-D multi-user virtual worlds for STEM programs based on the overviewed studies can assist educators and scholars to understand how they: a) can bridge effectively learning concepts with the practical consequences of students’ actions promoting collaborative problem-solving opportunities in realistic or vital simulation-based environments; and b) can facilitate the efficacy of practical hands-on experiments in contemporary instructional formats (online or blended).

The most important aspects from the aforementioned are provided in a workflow created by integrating educational potential of 3-D multi-user virtual worlds, according to the reviewed studies in furtherance of conceptualizing pedagogical instructional design principles. Fig. 2 portrays an illustration of a workflow that gives an overview of the design process to answer RQ4. According to studies that were reviewed previously, four peripheral elements (learning foci, instructional design, the utilization of 3-D multi-user virtual worlds and the theoretical foundations), two methods (content design and learning by different instructional design methods), and six steps in the teaching process based on two subjects (students and instructors) contained within it. These distinctions can help educators and instructional researchers to get a practically-infused instructional workflow for better understanding the propositions of getting involved in various learning tasks of STEM educational disciplines via 3-D multi-user virtual worlds.

Fig. 2
figure 2

A proposed instructional design workflow

Full size image

With the available built-in tools and communication channels that 3-D multi-user virtual worlds can deliver, several practical issues of their pedagogical and effective exploitation in STEM courses are raised. Educational activities and results of the evaluation from these activities that presented in this review provide a novel path aimed at better understanding for teaching and learning activities, so that all users can utilize 3-D multi-user virtual worlds efficiently in their practice-based tasks and understanding fluently the main instructional and technological affordances affecting the construction and development of a virtual learning environment. The main goal could be the promotion of users’ experiences which can be applied in a realistic simulation-based environment or teaching and learning activities that mimic those of a “natural ecosystem”, like activities that organized into classrooms and/or experiments outside the conventional school schedule (Beltrán Sierra et al. 2012; Keskitalo et al. 2011; Merchant et al. 2013).

5 Conclusion

The use of 3-D multi-user virtual worlds can overcome the irreversible changes of classical laboratory training sessions, and supplemental phenomena which are not possible to consider due to flexibility, minimal cost for experiments or damage risk issues. Another possible role is to use them as preparatory environments where students can start their training in a virtual laboratory, with all the advantages (high volume, low cost, security, remote access, enriched digital environment), by following different instructional formats aimed at further strengthening their professional path with more experience and confidence in the real laboratory equipment. Pellas (2014) have emphasized the importance of collaborative learning using built-in tools and artifacts via 3-D multi-user virtual worlds beyond the accomplishment aspect and the characteristics of users’ interactions, taking under consideration instructional design frameworks that lead to differentially joint efforts. The main recommendation based on this study finding concerns the important role of collaboration based on theoretical foundation of Socio-Constructivism. Creating the conditions for students to discuss and exchange their opinions provide them with the appropriate learning environment to explore and encourage an investigative attitude which are fundamental features of an instructional design aimed at supporting knowledge acquisition through interaction. In addition, by using 3-D multi-user virtual worlds, all tasks formulate in such a manner that require from students to capitalize on this interaction, and to occur their opinions, or contributions into objects (artifacts) that are dynamic. A type of interaction like this one may need a collaborative climate to grow under a specific instructional design guidance underpinned by (socio-) constructivist-oriented learning theories in order to recognize all users the appropriate management responsibilities to succeed the learning objectives. The assessment results from previous studies have shown the positive impact of this technology on students’ learning outcomes (knowledge transfer including cognitive, higher-order, problem solving and social skills) and their engagement (affective learning experience) to be increased at a large extent.

The present mixed method systematic literature review has indicated that 3-D multi-user virtual worlds can become candidate platforms for STEM education. This review provides a useful guidance for educators, instructional technologists and researchers in the area of education, which can be supported by 3-D multi-user virtual worlds. The contribution of this study is also through:

  1. (a)

    the critical overview of the development and practical insights of prior studies related with 3-D multi-user social, open source and collaborative virtual worlds from 2000 until the first quarter of 2016 with a 13-year review of the literature,

  2. (b)

    the description of recent studies that utilized 3-D multi-user virtual worlds in different disciplines or learning scenarios from Primary, Secondary to Higher education are also worthwhile and should be mentioned,

  3. (c)

    the relevant description of exploring the use of 3-D multi-user virtual worlds in different STEM disciplines and better understanding of human-computer interaction through visually-rich content design characteristics that can become useful for another future work,

  4. (d)

    the understanding of how previous studies utilized main features of 3-D technology in a multi-user environment (e.g. the sense of co-presence or the sense of psychological immersion) can support further outcomes or tasks in STEM education to counterbalance students’ engagement in alternative and valuable participatory ways by utilizing different instructional formats, and finally

The significance of the current mixed method systematic literature review provides some interesting directions and guidelines in instructional technology research and development about the nature of the instructional methods that previous studies have implemented and evaluated for their effectiveness using 3-D multi-user virtual worlds. Results of previous studies suggest some recommendations for best practices in online or blended course delivery methods in STEM disciplines, which are included in this review. It is arguably accessible for recommendation that the researchers should develop and seriously take into account the appropriate theoretical underpinnings based on contemporary learning theories for binding students’ engagement and participation. All these processes can really assist students’ motivation and learning, as well as the instructional framework of communication and social interaction in a common virtual platform.

In closing, the utilization of different instructional design frameworks, design learning approaches in case studies and empirical evaluations using 3-D multi-user virtual worlds for STEM education, identifying as well various tendencies and shortcomings useful for future research. At the same time, the structure of this review analysis can serve as a guide for other researches by providing the following instructional design decisions to consider when designing virtual learning environments (VLE) in 3-D multi-user virtual worlds: a) instructional design decision of learning goals and under what theoretical underpinnings should all activities be conducted, b) instructional design decision of participants taking under consideration the educational experience framed and users’ prior experiences, c) instructional design decision for the implementation of courses in realistic or game-like settings which can support effectively learning contexts (interaction design), in which instructors should consider how the environment should be used, and which instructional and technological affordances can be supported efficiently for getting knowledge acquisition easier all students, d) instructional design decision for the number of all users, which patterns of group interaction can be implemented considered the emergent social interactions that can be provoked meaningfully, e) instructional design decision for the orchestration of users’ actions in a persistent 3-D environment, in which the role of each one and what aspects or perspectives seemed to be crucial for their engagement and participation; f) instructional design decision for the learning schedule, in which all activities should be mapped adequately related with learning goals using built-in and a/−synchronous communication tools.

The educational potential of 3-D multi-user virtual worlds are worthy to known, because of the benefits (affordances) influencing students’ cognitive acceleration and increase their self-management or enhance their engagement in practice-based activities. It is expected that this review can become useful for instructional technologists and educators to acknowledge the educational potential and affordances of 3-D multi-user virtual worlds in different STEM disciplines and expand as a guide toward using them in the future. The increasing progress in computer graphics and 3-D multi-user virtual worlds technologies can provide the opportunity to rapidly increase the use of virtual laboratory based systems applications following blended and online instructional formats, and due to the technological infrastructure that is provided can eventually reduce the need for real world laboratories altogether. Future studies must continue to define researchable variables, and researchers must adhere to sound methods of analysis. Also, triangulation of data is encouraged beneficial to exemplify the effectiveness using instructional deign models in 3-D multi-user virtual worlds.

6 Implications for research and practice

Several implications for research and practice can be extracted. The present literature review reveals that comparatively few studies provide evidence about the learning effects of 3-D multi-user virtual worlds use in content design (Beltrán Sierra et al. 2012; Okutsu et al. 2013; Xu et al. 2011), or performance evaluations about the effects of instructional design frameworks in realistic simulation-based conditions (Burgess et al. 2010; Merchant et al. 2013; Pellas 2014; Vosinakis and Koutsabasis 2012). The authors are aware that virtual laboratories and practical hands-on experiments created in 3-D multi-user virtual worlds are often used as an initial step in a student’s engineering education and training, followed by more in-depth hands-on experience with real authentic equipment using built-in and a−/synchronous communication channels. However, the analysis of reviewed studies has performed that this technology can accommodate not only a variety of contextual, practical experiments and pedagogical approaches, but also highlights the need for further longitudinal for the existence of design tensions making the conception, implementation and appropriation of these “worlds” still a challenging task.

The typology of educational potential and instructional affordances (i.e. pedagogical-socio-cognitive, technological-operational and organizational-financial, see Fig. 1) in combination with the instructional design workflow (see Fig. 2) that proposed in this study are expected to contribute both research and practice in STEM education. Following this typology, different factors influencing the results of 3-D multi-user virtual worlds which support learning at different fields of STEM education are systematically investigated and discussed. More studies examining students’ participation and effectiveness of these “worlds” for practical hands-on experiments contrary to other 2-D learning platforms should be validated further.

From an instructional perspective, this study hints that interactive activities using a 3-D common multi-user environment can be designed and supported by adjusting the nature and complexity of different STEM tasks through interdisciplinary instructional contexts, only by tailoring the management responsibilities of each team, according to students’ needs or demands. When designed in a targeted manner, such technologies should involve the type of tools and functionalities that support joint work on knowledge content, following up and tracing the co-constructed knowledge in a consistent manner.

From an educational technology and design perspective, the compliance of using 3-D multi-user virtual worlds to overcome the classroom or laboratories constraints is truly the consequence of long and careful co-design leaning processes. Of course, there are still remain some concerns that may constrain the full realization of distance education in STEM education because of the realization that inevitably laboratory (hands-on) exercises require. In these demands, educators and scholars should consider how can better develop a virtual laboratory with distance access to connect remotely students who are spatially separated. The option of creating a remote-access laboratory, although it can useful for supplementary courses in a persistent manner, especially in relation to the communication and sensory-control hardware and software required and the overall expense of the equipment and maintenance remain in low levels. Also, it can become a relatively efficient solution with high scalability (many students can access in multi-user environment each time that they want), supporting to this notion more instructional design methods for the implementation of CSCL scenarios through online or blended instructional formats. The configuration and development of a virtual learning environment for STEM courses is not quite difficult to design and prototype, but it is widely more difficult to design an instructional framework that let end-users (instructors and students) become more appropriate to them. The advances in combining web-based technologies with 3-D multi-user virtual worlds, such as Sloodle (Livingstone et al. 2008) or Scratch for Second Life (Scratch4SL) can low the barriers during students’ first time entry, reducing adequately the “steep learning curve” in blended or fully online course delivery methods (Pellas 2016).

From an educational-practice perspective this review covers certain remaining challenges in this field, representing interesting directions for STEM education: a) the design of a learning space using a 3-D multi-user virtual world is identified by the advantages of persistence and adaptability form factors, which have sparsely limitations, indicating potential paths for the design of novel systems; b) the increased length of co-design processes in the direction of addressing real classroom’s or laboratories’ constraints, and the wide range of applicability using built-in or a−/synchronous communication tools point towards the need of specific instructional design guidelines and processes for the conceptualization and implementation of learning scenarios in 3-D multi-user virtual as candidate VLE; c) the need of addressing the design, preparation and customization of the learning activities before enactment itself is also unresolved, as it often requires quite specialized technical knowledge and careful interaction design.

From a research and instructional design perspective, 3-D multi-user virtual worlds present certain design tensions in applying VLE-based systems for the requirements of the class supported studies: a) within a flexible and adaptable environment according to STEM courses’ needs, users’ engagement and participation can quickly become valuable to illustrate even the most complex experiments; b) the exploitation of an instructional design framework make these “worlds” unique resources for virtual classroom supported settings using different instructional formats (blended/online), in contrast of having one-to-one 2-D digital oriented setups; c) the use of different constructivist-oriented instructional formats and instructional design frameworks are the main affordances for having all users contribution to the learning content and observe the consequences of their actions in a common virtual environment; d) the ability of 3-D multi-user virtual worlds to provide automated feedback is somewhat not hampered, if users want to use their technological capabilities as a flexible input digital platforms for practice-based learning tasks. Such an approach can indicate several tensions by highlighting pathways that must be explored in future research work, by conducting comparative studies in different classroom settings.

7 Limitations

The results of this literature review indicate the following limitations:

  1. (a)

    The instructional design methods or models that have been presented in conferences, book chapters, symposiums or workshops were not included.

  2. (b)

    This study did not provide extensive technological or instructional boundaries, like in their studies Gregory et al. (2015) and Coban et al. (2015) have addressed. Several barriers in using 3-D multi-user virtual worlds as a mainstream teaching tool and their failure to maintain their popularity are issues that should be analyzed further, especially for STEM courses.

Table 6 The specific protocol executed in each database
Full size table