Keywords

1 Introduction

Education institutions realise the value of games as a pedagogical tool, considering their increasing sophistication in terms of technology, game mechanics, visual imagery and communicative elements. According to the literature, pedagogic games can harness students’ motivation, engagement and persistence, teaching complex disciplinary concepts that can be retained in memory for a longer period of time [1, 2]. Recent studies also highlight the educational value of simulation games for student learning, arguing that such games could provide students with practice grounds that are related to real-world contexts [3].

This study examines the use and pedagogical potential of Cities Skyline in an Urban Planning module. City Skylines is an open-world city building simulation game where players are involved in urban planning and governance. Players are tasked with developing small to functional mega-cities by developing roads and infrastructure, zoning districts, utility management, public transport, taxation and implementing civil policies. Amongst others, players are also involved in the management of the city’s economic performance and control elements of health & safety, education, employment levels, pollution and sustainable growth. Additionally, the game provides enormous creative freedom in its sandbox mode and has several mods/expansion packs to augment the development of superior cities.

Research has investigated the potential of Cities Skyline as an educational tool in a wide range of fields, such as real estate studies [4], environmental sciences [5] and journalism [6]. However, an insufficient number of empirical studies regarding the educational benefits of Cities Skylines in urban planning education remain; specifically, there is an apparent lack of studies that evaluate the benefits of Cities Skylines from the critical lens of learning theories.

Informed by Kolb’s Experiential Learning model [7], this paper aims to evaluate the effectiveness of the simulation game Cities Skylines for teaching Urban Planning discipline from the perspective of students. It focuses on the lens of learning outcomes, namely, cognitive, behavioural and affective outcomes.

2 Literature Review

2.1 Games in Education

Games allow users a constant form of “interactivity” that feeds their gaming experience when they receive valuable feedback corresponding to their actions as the gaming narrative unfolds. The ability to control their environment and characters enables users to utilise control mechanisms to overcome challenges they may face. It also allows profound associations and sensitivities with the characters that fuel their identity as players. Most importantly, games allow players a form of immersion and integration within the game; being present in a make-belief world encircled by challenges is arguably the highest pleasurable gaming aspect [8, 9]. Educational games connect these elements into an alternative universe and disciplinary scenarios enabling users to relate to the character’s circumstances within the game. Developers then provide all the necessary tools to players that are required for them to comprehend their distinctive situation and their connections in real-life situations [10]. Various disciplines have made profound use of this teaching method to validate its use in pedagogy [3, 11,12,13].

2.2 Effective Features of Educational Games

There are certain features of critical importance in games to be considered as useful tools for learning. These features are the main component for learning requiring “deliberate intertwining with pedagogical content to ensure successful learning” [14, p. 31]. These were identified as the Back Story & Production, Environment & Realism, Adaptivity and Interactivity & Debriefing [14, 15].

Backstory and Production.

The narrative structure, which is regarded as the “sticky element”, is closely associated with the immersiveness of computer games; players’ interests remain consistently high while exploring in-game complexities. The “narrative-pedagogical mechanism” is the first contact between the game and the wider world, allowing players to interact with gameplay, in-game elements and fellow players [16].

Moreover, learning and knowledge, which ought to be carefully considered, is embedded within the games [17]. The immersive function, however, is not a promising element for learning, where irrelevant material or over-doing narrative can be counter-productive to learning processes [18].

Environment and Realism.

An environment of realism is directly proportional to the skill level of students and their experiences; the higher the fidelity (realism) and complexity the game reaches, the greater learning opportunities there are for experienced players. Notably, with a high learning curve in realism, players feel more challenged and motivated [19]. For mature students, realism is a persuasive feature for learning-simulations at tertiary education, thereby giving it greater approval for learning [14].

Adaptivity and Interactivity.

Fundamentally, the knowledge tool must adapt to the learner’s abilities and preferences for effective learning given the individual differences in learning capabilities [20]. Researchers have identified several adaptive features for in-game systems, including catering to one’s preferred presentation and format of the information, enabling them to see “different page content of the same knowledge point” [21, p. 163]. Specifically, researchers argue that game systems should monitor players’ learning progress, i.e., areas mastered in the form of a “knowledge tree”, provide relevant links to areas from where the learner can acquire related information/fundamentals subject to accomplish tasks. The interactive function of feedback and engagement with the game systems is central to transferring knowledge between the game and the learner. Research suggests that interactivity features could bring out positive effects on a student’s feelings of social presence [22].

Debriefing.

As part of the evaluation framework, debriefing covers the knowledge sharing practices between students based on their experiences in the game. Debriefing enables knowledge sharing practices to accomplish tasks and increases gaming engagement levels. These happen through progress tracking, animated agents, discussion forums, live-chats. Crookall emphasises this point, stating “learning comes from the debriefing, not from the game” [23, p. 907].

2.3 Experiential Learning

Experiential learning is defined as learning which “exists when a personally responsible participant cognitively, affectively, and behaviourally processes knowledge, skills, and/or attitudes in a learning situation, characterised by a high level of active involvement’’ [7, 24]. Previous research demonstrates that experiential learning can bring out better conceptual understanding, critical thinking and problem-solving skills, increased enthusiasm, better performance, a higher level of self-confidence and learning enhancement [25].

Experiential learning fosters several learning outcomes that could guide researchers in measuring effectiveness in educational games. Ranchhod identifies them as: Cognitive Development (conceptual understanding): where students internalise concepts, terminologies and core principles of the discipline. Behavioural Development (skill development): The realistic representation of the world enables students to transfer their acquired skills to real-world situations. Affective Development (Evaluation): “Skills development has a positive impact on students’ affective evaluation of the simulation game exercise”, enhancing students’ engagement and satisfaction and improving their attitudes towards the discipline [25].

3 Methodology

This research adopts a qualitative approach based on the research philosophy of interpretivism, in which “interpretive researchers assume that access to reality is only through social constructions such as language, consciousness, shared meanings, and instruments” [29, p. 38]. The semi-structured interview method was used to gather data from a sample size of eight participants. Each interview was 45 min in duration.

3.1 Participants and Contextual Background

Participants were post-graduate students (ages 22–27 years). Seven of them were part of an Urban & Regional Planning program, while one was studying at a Real Estate program at a British University. They had taken the Sustainable Development module, in which they were introduced to the Cities: Skyline game. Three participants had completed undergraduate degrees in Geography, while the remaining undergraduate participants (five) had one to five years of work experience and studied in one of the following disciplines: Architecture, Electrical Engineering, Biosciences, Urban Planning, and Business Management. The students were not notified of the game being introduced as a teaching tool in the module, so it did not influence their decision to take the module.

The students were taught about the concept of sustainable development, which is complex and difficult to comprehend. To address this, each student was required to explore sustainability by developing their virtual cities according to concepts learnt in class, but using real-world policies as a basis. They were then encouraged to broadly analyze their concept with another student. This allowed the students to form a balanced understanding of the otherwise ambiguous concept of urban sustainability.

The learning goal of the module was to prepare students to write sustainable development policy papers as their final assignment. The game on its own does not have any direct link to writing policy papers. It merely serves as “a backdrop to things learnt” (Respondent 3) and provides a learning/supporting environment for the implementation of key concepts to be reflected on. Based on these reflections, policy papers are written.

Cities: Skyline is a single-player, off-the-shelf game and, therefore, restricts collaborative learning due to the absence of networking. To overcome this constraint, the class instructor grouped students into pairs to enable knowledge sharing before further guiding them on the practical implementations of their learnings within real-life contexts.

Two classroom sessions were dedicated to training students for setting up and installing the game. The instructor played a pivotal role in familiarizing students with the game interface and teaching them the in-game tools. Students were introduced to the five different build-types/zones: commercial, residential, industrial, municipal, and office. The students were divided into pairs and tasked to use the game as a “canvas” for building cities according to the sustainable development concepts and theories learnt in class.

Students learnt sustainable-development concepts in class and gameplay took place both in and outside of lecture sessions. There was no time limitation on students playing the game; however, to reduce the learning curve and keep the game interface simple, the vanilla version (original game with no expansion packs) was utilized. While playing, students simply monitored how the population in their cities evolved and responded to their city building policies as their cities grew over time. Based on this information and the in-game restrictions (what the player wanted to do versus what the game allowed them to do), students reflected on what they learnt and their overall experiences in the following in-class Q&A session.

Students discussed their experiences with the instructor to deepen their understanding of concepts, which included discussing in-game limitations to find alternative approaches. The instructor played a key role in instigating debate and discussion among learners to critically reflect on their solutions and ideas of what they may do in realistic situations. Students would also consider improved methods of achieving sustainability and, under the guidance of the instructor, debate on why certain policies are effective in specific real-life scenarios.

3.2 Semi-structured Interviews

Data Semi-structured interviews were conducted in a conversational manner [30]. This method was undertaken to ensure the interviewees were relaxed and able to share as much as possible from their experience. These conversations were intended to open up new subject areas and help reveal valuable information leading to high value data [31]. The researcher had no experience with the game nor did they have any relations to the participants. The interviews were audio-recorded and later transcribed. This research has received ethics approval and informed consent was collected prior to all interviews. When recording the interviews, respondents were assigned alias names.

The questions used to structure the interviews were devised through “inventory of learning goals” categories, which were previously used for a marketing strategy simulation game [25]. The current paper uses the same categories to devise questions related to the experimental learning of Cities: Skyline. The four categories of the interview questions are as follows:

Within the Learner’s Background category, questions were geared to identify each learner’s specifications, which includes the learner’s age, previous education, and work experience. Questions in the Cognitive category were geared towards conceptual understanding, which included understanding sustainable-development theories, core concepts, and terminologies. (Example Question: “How has the game assisted you in learning core Urban Planning concepts and principles? Please name a few.”) Questions within the Behavior Implications category were aimed to find out about the game’s influence on fostering skill development. (Example Question: “Has the game facilitated in you Urban Planning creativity? If yes, give some examples.”) And lastly, questions within the Affective Evaluation category were aimed to identify the game’s influence on emotional aspects as an effective learning tool. For example, whether learners did or did not feel highly involved and engaged in their learning experiences. (Example Questions: “How has the game contributed to your interest in the Urban Planning field; how has it influenced your motivation and engagement level (if it has) in the learning process?).

A pilot study of the interviews was conducted to allow the interviewer to familiarize oneself with the script flow and understand how the conversational style of the interview would allow for further questioning [32]. This also helped the interviewer determine the interview duration, so that it would remain within the timeframe noted on the consent form. The test also helped identify the relevance of the questions and helped the interviewer determine which questions could benefit from an explanation or an example to provide clearer understanding to the interviewee.

3.3 Data Analysis

The data was analyzed iteratively by researcher through the inductive approach by using a Thematic Analysis approach [26]. This involved finding commonalities and repetitions which are identified by comparisons of interviews and themes are identified. This method is useful as it provides flexibility and is suitable for new research interviewers. It is appropriate for analysing and summarising masses of in-depth raw-data and is convenient for highlighting similarities and differences between them. It is popular for psychological interpretations/analysis of raw-data and allows generation of unanticipated findings.

Thematic Analysis was done in the following stages:

  1. 1.

    Familiarization with research data.

  2. 2.

    Insertion of line numbers to function as markers.

  3. 3.

    Categories (Cognitive, Behavioral, Affective) assigned a distinct colour code.

  4. 4.

    Coloured coding applied to data, where it was felt that respondent touched upon a subject directly or indirectly relating to a specific category in his/her responses.

  5. 5.

    Data was then inserted under the similar color-coded Category Tables.

  6. 6.

    Themes were identified and marked as Dominant Theme or Sub-theme accordingly.

  7. 7.

    Data was re-analysed, and similar themes were extracted and inserted into Themes Tables.

4 Results and Discussions

The results focus on key areas of student learning experiences. Following the experiential learning model, the results are presented through three lenses: Cognitive, Behavioral, and Affective learning outcomes.

4.1 Cognitive Development (Conceptual Understanding)

Analysis of data suggests that although Cities Skylines could enhance student conceptual knowledge on sustainable development, it lacks subject related terminologies and guidance from the teacher. Research indicates that simulation games allowed users to implement and test several geographical concepts that assisted in their understandings [3, 7]. This research acts as an extension to that research, where students took the opportunity to implement an array of sustainability concepts in Cities Skylines. This augmented their conceptual-understanding, as the game helped them visualise and monitor its effects in a simulated-world. Consequences encountered based on learner’s decisions and designs in the game also complimented student learning outcome.

The theory comes hand in hand with the implementation of the game (…) It is a tool to give me more insight about the concept that is delivered (…) This game gives you an understanding about sustainable development [Participant 1].

Learning new terminologies from the simulation is an essential indicator of pedagogic effectiveness [25]. Cities Skylines did not provide convincing results for introducing new urban-planning/sustainable-development terminologies. It used simple English terms to help students to develop a general understanding of those terms.

When I was making green cities, Green is a term that is not new for me, so when you put it into a working city, it is a whole different thing [Participant 1].

Our data suggest that the instructor’s role becomes central, as students who failed to recognise this method for learning, had a negative experience of using the game. This is consistent with the literature [27, 28], suggesting the instructor needs to play a vital role as a facilitator in the learning process to draw out the maximum pedagogic potential of games. This highlights the importance of the instructor in providing a debriefing on how learning would take place and keeping close attention to student’s learning process.

The game does not connect to the concept because of its own set of principles (…) The game has set existing policies that have to be used – if I disapprove anything within the game, I have to go along with it [Participant 2].

A real challenge for students was to accurately reflect on the connections they could make with the game and its application in real life. The game, however, does not prepare students for its applicability in real-life situations due to its built-in limitations. For example, participant 6 stated that they wanted to implement car-free zones, but it was simply not achievable in the game. Here, the course instructor role’s remains central in instigating debate and discussions amongst students to critically reflect on their ideas of what they ought to do in realistic situations.

The critical process of having an idea of what you want to do with your city and not being able to do it, allows you to analyse what you might do in real life situation (…) It allows you to understand and apply the concepts however in real life on a lot of complex and wider scale [Participant 3].

In general, the game functions as a “canvas” for testing taught concepts. It is not a self-autonomised tool for learning concepts and terminologies, though it does demonstrate the profound potential for further development. This tool requires close facilitation by the instructor to ward off impacts of limitations, to achieve maximum pedagogic benefits. Therefore, it has been relatively successful in delivering conceptual understanding. It is instrumental for knowledge retention; however, it cannot replace training for real-life situations. Majority of participants advocated its use as having helped them to feel confident about their learning.

4.2 Behavioural Development (Skill Development)

Data analysis suggests that students value Cities Skylines for enhancing their critical thinking and problem-solving skills. Although creativity was also frequently mentioned by students, it could not be easily transferred into real-life contexts. In addition, students felt the built-in limitations of the game does not encourage collaborative learning.

Students identified certain knowledge-gaps that the game had helped them fill as they pursued their sustainable-development goals. Countering traffic problems was one of the significant challenges for some students. They had learnt how to critically evaluate the situation and design practical solutions, notably paying close attention to the order of functioning of traffic lights and the need to consider quality-of-roads, weather, residents nearby as opposed to emphasising widening roads to counter traffic issues. This is consistent with the literature which suggests that simulation games could promote student critical thinking skills [33].

In case of traffic congestion, you need to critically think on how to solve the traffic problem in the area. When you build new settlements or residential areas you are thinking what the implication is it would have on my transport, am I going to need more schools, healthcare facilities, more services. So, it’s a critical process [Participant 3]

According to students, the game provided them with a risk-free zone to be creative in designing urban cities. Nevertheless, they were not sure how these creative solutions could be transferred into real-life contexts.

Planning cannot happen without inclusion of stakeholders (…) community would create like an action group against my development or vote against my development. You can’t just make a road or a school; you actually have to go through the political process, get planning permission and go through public engagement, whereas there is none of that in the game [Participant 3].

As Cities-Skylines was a single-player game, students revealed the need for improved collaboration elements, such as multiplayer, forums, discussion groups, video display and voice chats.

I’m not sure if it has chatrooms on it but I didn’t see anything multiplayer or anything Because I think we just go guided on a solo play environment and I never played the game before and you had one account and you had to do a new game from there [Participant 8]

Overall, Cities Skylines generates specific skills for students; major ones include critical thinking and problem-solving skills. It also delivers in fulfilling certain knowledge-gaps in specific areas, where students experience unexpected consequences based on their decisions within the game. The game also addresses key matters related to districts, zoning and public services. Students highly approve of its skill generation, but they find an immense gap as a medium for applying acquired skills in real-life contexts.

4.3 Affective Development (Evaluation)

In this category, results showed that most students were highly motivated and engaged as they encountered the course material through the game. This is consistent with the existing research which reveals that simulation learning enhances trainees’ affective and cognitive processes [1, 33].

However, improvements are needed to align student game experience with their learning objectives. Several participants reflected on a change of perception of learning, with increased interest, demonstrated that Cities Skylines is a valuable tool to aid traditional learning methods. This resulted in most students demonstrating immense satisfaction derived from the learning process.

This is the first time I have experienced being taught theory using a game, I think this is a very interesting approach and a good way to do it. For motivation I will give it an 8/10. The game is very engaging; I would give a 9 or 9.5/10. It is a very demanding game [Participant 1]

However, the absence of a back story (debriefing) limited students’ ability to connect game experience with real-life contexts. This shifted enormous responsibility on the instructor to stir the game experience and structure the lesson to bridge this gap.

But I didn’t know how this game was going to relate to the module [Participant 2]

Contrary to the results of existing research, which emphasize the role of computer-aided instructions (CAI) in the learning process [34, 35], our research suggests that CAI is not adequate as a sole instructional tool. The role of the instructor is of paramount importance to extract the pedagogic benefits and fend off in-game restrictions.

There is a tug-of-war of interests there because, if I do not comply with the notification (instructions) of the game, something will happen to your residents ... they will leave your city [Participant 1]

In summary, positive developments were identified in terms of students’ conceptual understanding, skill development, motivation, and satisfaction, which also effected their attitudes toward the subject. Nevertheless, the instructor’s role is of paramount importance to keep the game in check, to prevent students from veering away from the learning objectives, and to extract maximum learning outcomes from the game.

5 Conclusion and Recommendations

This study reveals the pedagogical and phenomenological aspects of the simulation game, Cities: Skylines, for the urban-planning discipline. Eight in-depth interviews were conducted with postgraduate students from the sustainable development module to explore their views on the effectiveness of the game. The findings provide empirical evidence that Cities: Skylines can be an effective pedagogical tool for enhancing students’ conceptual knowledge, while increasing motivation and satisfaction regarding the learning process. The research results suggest that elements of the game could help develop specific skills relevant to urban planning disciplines, including critical thinking, problem solving, and creative thinking. To be considered an effective educational tool, knowledge sharing and interactivity are of critical importance. Hence, it is recommended that game developers design educational games with a multiplayer option in order to enhance networked interactivity. Research limitations revealed gaps in the development of subject-related terminologies and the transferability of knowledge to real-life contexts. Thus, this research highlights the need for aligning game experience with the learning objectives of the module. This research also acknowledges the essential role of the instructor in bridging the gap between knowledge and skills acquired within the game and in real-life contexts. Future research would benefit from a larger sample size or a mixed-methods approach. Future research could also involve testing using the game Cities: Skylines - Green Cities, which is a recent expansion showing promising predisposition for teaching subjects related to sustainable development.