Keywords

1 Introduction

An artificial intelligence-based chatbot, ChatGPT, was launched in November 2022 and can generate cohesive and informative human-like responses and engage in natural language conversations with users. In higher education, ChatGPT has sparked both excitement and scepticism. On the one hand, it is believed that ChatGPT will inevitably become an integral part of higher education, and therefore some education practitioners recommend involving students with such tools to enrich the learning experience rather than prohibition (Farrokhnia et al. 2023; Rudolph et al. 2023). In this context, it is believed that Chat GPT can help students develop different skills, such as reading, writing, information analysis, critical thinking and problem-solving, generating practice problems, etc., being relevant in public administration practice and research (AlAfnan et al. 2023; Rahman & Watanobe 2023; Rudolph et al. 2023). On the other hand, it is believed that ChatGPT will bring multiple challenges, such as copyright issues, bias, fairness, excessive reliance by students, data privacy and security, etc. (Farrokhnia et al. 2023; Lo 2023; Rahman & Watanobe 2023). Although some existing studies provide initial insights into the potentials and challenges of Chat GPT, they do this from a teacher rather than a student perspective.

Early on, after ChatGPT was offered to the public, Wagholikar et al. (2023) found very important results in their online survey study that, due to its ease of use, awareness of ChatGPT did not differ between genders or between fields of study. While the ChatGPT is seen as a disruptive element in education, innovative teachers will know how to use it to their advantage and the benefit of their students (Rudolph et al. 2023), as it can be used in education to enhance teaching and learning for both teachers and students. For students, ChatGPT could act as a tutor and support their learning, for example, by answering questions and summarising information (Lo 2023; Flores Limo et al. 2023; Rahman & Watanobe 2023). ChatGPT tutoring is a form of one-to-one tutoring where advanced technology allows instant interaction, so students do not have to wait for a tutor or teacher to answer their questions. The results of the survey reported by Flores Limo et al. (2023) showed that almost 70% of the participants used ChatGPT as a tutor for their learning. The results also showed that satisfaction after the use of ChatGPT, self-confidence and the motivation to learn were significantly related to the length of the individual tutoring session. Even more interesting is that learners can argue with ChatGPT about the explanations, solutions and other suggestions given, and other suggestions (Rahman & Watanobe 2023), developing competence in argumentation and critical thinking.

ChatGPT can also facilitate collaboration by generating different scenarios for students to solve problems together and provide personalized feedback and guidelines (Lo 2023; Rudolph et al. 2023). ChatGPT can also help with writing skills by providing grammar corrections and suggestions (Lo 2023) or even assist the student by providing them with the structure of the essay (AlAfnan et al. 2023). Similarly, the strengths of ChatGPT, such as the generation of plausible answers and the provision of personalised and real-time answers, were identified by Farrokhnia et al. (2023). They saw opportunities in the increased accessibility of information and the facilitation of personalised and complex learning.

The study conducted by Ali et al. (2023) between English language learners investigated motivation during learning. The research shows that the involvement of ChatGPT in language learning was generally a motivating factor, especially in reading and writing, while it wasn't perceived in listening and speaking. The authors conclude that the new AI general-purpose conversation chatbot is recommended to be a part of English language learning, as it motivates learners to learn autonomously and should not be avoided in the learning process. Similar is reported by Sotelo Muñoz et al. (2023), who have found that ChatGPT is supposed to increase students’ motivation and engagement in the learning process. With motivations to read and write more than other learning processes in English language learning.

Shoufan (2023) presented a survey conducted among senior computer science students to assess the potential of ChatGPT and its impact on teaching and learning. In the first phase, they evaluated ChatGPT in an open-ended question after having used it in one of the study activities. In the second phase, they had a longer period of using ChatGPT for prepared activities, and after that, they answered a questionnaire. Students rated ChatGPT as easy to use, motivating them to work. The students were impressed by ChatGPT's capabilities and emphasised the human-like interface with clear and structured answers, and expressed positive attitudes. However, they also pointed out that the answers are sometimes not precise, concerned about manipulations and malicious use and exposed that ChatGPT could not substitute human intelligence.

However, to fully understand the impact of ChatGPT on higher education, it is crucial to understand students’ experiences and perceptions of it. Namely, students’ perceptions are highly relevant for educators as they can significantly impact their motivation, engagement and academic achievement. Therefore, the paper aims to examine public administration students’ perception of ChatGPT. Specifically, it is focused on students’ perceptions of ChatGPT's capabilities, study issues within its context, and its ability to facilitate skills development and enhance outcomes. The rest of the paper is structured as follows. The next section presents the data and research methods used in the study. The final section provides a conclusion, summarising the main findings and implications.

2 Data and Research Methods

The data were a part of the ChatGPT Student Survey entitled “Students’ Perception of ChatGPT”. The online questionnaire was designed in English and consisted of closed-ended questions covering different aspects relevant to the opportunities and challenges of ChatGPT for students. The developed online questionnaire consisted of selected socio-demographic characteristics of the survey participants and several important aspects/sections relevant to the opportunities and challenges of ChatGPT for public administration students: 1) students’ awareness, focusing on how they are familiar with ChatGPT; 2) students’ knowledge, concentrating on the technical aspects, recent advancements, potential implications and limitations of ChatGPT; 3) ChatGPT capabilities, such as its ability to understand and respond in human language, improve various industries, and perform certain tasks better than humans; 4) students’ concerns, focusing on the potential negative consequences of ChatGPT, such as job loss, misuse, the spread of misinformation, and invasion of privacy; 5) students’ satisfaction, focusing on their experience interacting with ChatGPT; study issues from a student and teacher perspective; 6) implications for the labour market; 7) regulatory and ethical considerations; and 8) usage potential of ChatGPT in improving learning and working. Individual aspects were measured on a 5-point Likert rating scale ranging from 1 (e.g., strongly disagree) to 5 (e.g., strongly agree).

The survey was intended for all higher education students enrolled in public administration study programmes in Slovenia. The data collection process took place between May and July 2023 and was facilitated through various communication systems and social media to obtain a representative sample of public administration students. The empirical analysis relies on descriptive statistics, which unveil the mean values of perceptions among the students, encompassing various aspects of ChatGPT. These aspects encompass students’ perceptions of ChatGPT's capabilities, study issues within its context, and its ability to facilitate skills development and enhance outcomes. The mean values are separately presented for each individual aspect of ChatGPT through bar plots. The arrangement of these bars in a plot is such that they are organized in descending order of the calculated means.

3 Data and Research Methods

By the end of July 2023, 157 public administration students (students from the Faculty of Public Administration, University of Ljubljana, Slovenia) had participated in the survey. Since the students were not obliged to complete the questionnaire in full, the number of respondents across questions may vary due to the missing values. The socio-demographic characteristics of the survey participants are presented in Table 1. First, most of the students (66.5%) were female and enrolled in the first level of study (81.5%). Moreover, most of the students had a general understanding of what ChatGPT was and what it could do (36.9%) and even used ChatGPT before and had a good understanding of its capabilities (35.7%).

Table 1. Socio-demographic characteristics of the survey participants.
Full size table

Regarding public administration students’ perceptions of the capabilities of ChatGPT, the results show that they – as shown in Fig. 1 – on average tend to agree the most with the statement that ChatGPT has a wide range of capabilities (M = 4.1). This is not surprising if we consider that ChatGPT can not only answer simple answers but can also be used for more complex tasks such as writing music, planning holidays, writing summaries, job applications etc. As more than half (55.2%) of the responding students claim to be very familiar with ChatGPT and have used it extensively (19.5%) or at least have used ChatGPT before and have a good understanding of its capabilities, it is also not surprising that they have tested different ways of using ChatGPT and have come to the conclusion that it has a wide range of capabilities. Students, on average, also agree that ChatGPT can summarize and paragraph lengthy articles (M = 4.04), which can be attributed to the fact that students often must summarize lengthy articles or textbook chapters to solve different tasks or to prepare themselves for certain exams. Since attending lectures is not obligatory at the faculty, the students can find such a capability very useful also in situations when they did not attend lectures and therefore did not prepare notes. ChatGPT can assist them with that. On the other hand, students least tend to agree that ChatGPT can write poems in different languages and styles (M = 3.42) or can write music in any genre (M = 3.31). One possibility for such a result is that the students did not test the beforementioned capabilities, but further research should be done before drawing any concrete conclusions.

Fig. 1.
figure 1

Source: Own calculations based on ChatGPT Student Survey.

Students’ perceptions of the capabilities of ChatGPT.

Full size image

Regarding students’ perceptions of study issues in the context of ChatGPT, the results show that they – as shown in Fig. 2 – on average tend to agree the most with the statements that ChatGPT can help with their studies (M = 4.15) and can save them time for assessment (M = 4.15). Both perceptions are in line with the students’ perceptions regarding the capabilities of ChatGPT, which include the fact that students agree that ChatGPT can summarize and paraphrase lengthy articles. Students can therefore save time for assessment and use ChatGPT as a help tool with their studies. This is not surprising, taking into account some of the significant applications of ChatGPT in education which can be used by students, such as allowing students to ask better questions, giving straightforward responses, providing text analysis, crafting essays, boosting exam preparation, providing exact information, summarising large documents (Javaid et al. 2023). On the other hand, students out of the given possible statements, on average (M = 3.45), at the least perceive ChatGPT as being able to or providing personalized education. Since possibilities exist for students, such as receiving personalised feedback using ChatGPT based on their writing style to help understand and concentrate on the areas they need to improve, being provided with on-demand, live tutoring or for everything to be explained to them multiple times (Flores Limo et al. 2023; Javaid et al. 2023; Lo 2023; Rudolph et al. 2023) it is probable that the students did not use ChatGPT in such a way or that they did and were not satisfied with the level of personalization.

Fig. 2.
figure 2

Source: Own calculations based on ChatGPT Student Survey.

Students’ perceptions of study issues in the context of ChatGPT.

Full size image

Regarding students’ perceptions of the ability of ChatGPT to facilitate skills development, the results show that they – as shown in Fig. 3 – on average tend to agree the most with the statement that ChatGPT has the ability to improve students’ digital communication skills (M = 3.57). Some studies observe that tools such as ChatGPT can potentially reduce some of the skills that students should master, including critical, creative and collaborative thinking skills and creative writing skills (Shidiq 2023, p. 354). Why students chose to perceive improving digital communication skills as a top ability of ChatGPT regarding facilitating skills development can be attributed to the fact that digital communication with another person electronically through tools such as text messaging, messaging applications, e-mail, social media sites (such as Instagram, Facebook, Twitter, Snapchat etc.) is very close to them and they use them often on a day-to-day or better minute-to-minute basis. It is only natural that students have therefore tested or used ChatGPT as a tool to help with their digital communication skills, which, as they perceived it, improved through the use of ChatGPT. On a similar note of “communication skills”, a completely different result emerged regarding students’ perception of ChatGPT's ability to facilitate and improve students’ face-to-face communication skills (M = 2.19). Some authors see ChatGPT as a tool eroding face-to-face communication, empathy, and emotional intelligence, which could potentially fuel a fragmented society where AI language models supplant the human touch in our daily interactions (Ray & Das 2023, p. 1). Their views could become a reality if ChatGPT will not offer satisfying results in using or improving skills in face-to-face communications. In one podcast, an interview was done with ChatGPT where the algorithm was asked if we (humans) still need to learn how to communicate better now that ChatGPT exists, to which ChatGPT replied that while ChatGPT can provide information and generate responses, it cannot replace the human interaction and emotional intelligence that comes with face-to-face communication (Abrahams 2023). As it would seem for now, considering the presented students’ perceptions, there is little correlation between ChatGPT and the perception of it as a tool to facilitate and improve students’ face-to-face communication skills.

Fig. 3.
figure 3

Source: Own calculations based on ChatGPT Student Survey.

Students’ perceptions of the ability of ChatGPT to facilitate skills development.

Full size image

Students were also asked about their perceptions of ChatGPT to improve outcomes. Results (presented in Fig. 4) show that they, on average, tend to agree the most with the following statements regarding ChatGPT 1. improves students’ satisfaction (M = 3.62), 2. increases the chance of completing the study (M = 3.61), and 3. increases the chance of completing co-curricular activities (M = 3.61). Students, therefore, perceive ChatGPT as a positive and helpful tool to be used in their studies, preferably because ChatGPT can save them time, which would otherwise be used for certain simple tasks such as summarizing and paragraphing lengthy articles (Fig. 1) or certain assessments (Fig. 2). By saving time for simpler tasks, students can focus on more demanding aspects of their studies and still have time to complete co-curricular activities and therefore have a better chance of completing their studies (in time). Some other studies implied similar improvement outcomes of ChatGPT, such as improving the performance of students on exams, which is especially true for low-performing students and those who suffer under time constraints (Choi et al. 2023, p. 12). The improvement of outcomes in certain areas is perceived by students as lacking. Such areas of possible improvements of outcomes are especially 1. reducing under-skilling (M = 2.97), 2. resolving skills obsolescence (M = 2.96) and 3. improving students’ employability (M = 2.93). Students, therefore, do not perceive ChatGPT as a tool for improving their skills or something that would help with their employability. Why students, on average, disagree with ChatGPT as a tool to be used is a topic for further research. As for now, we can conclude that ChatGPT can be used to improve certain student skills, such as language and writing skills, to boost their exam preparation and prepare them for interviews by improving their communication skills (Javaid et al. 2023, p. 8 and 10).

Fig. 4.
figure 4

Source: Own calculations based on ChatGPT Student Survey.

Students’ perceptions of ChatGPT to improve outcomes.

Full size image

4 Conclusion

ChatGPT is perceived by public administration students as a tool that can enhance learning experiences and not as a disruptive tool. Most (92.1%) of the students have already used ChatGPT or are at least familiar with what it is and how it works. Students perceive ChatGPT as having a wide range of capabilities, such as summarizing and paragraphing lengthy articles, while they do not perceive ChatGPT as much as a tool for writing poems or music. This is understandable if ChatGPT was used for study purposes or is, in their perception, connected to study issues, while public administration studies are rarely related to writing poems or music. Students also perceive ChatGPT as a tool that can help them with their studies and save them time. Students generally have positive perceptions of ChatGPT in relation to different study issues, which implies that further impact of the use of ChatGPT in concrete public administration study issues is recommended to improve the learning process from the students’ perspective. Students also see ChatGPT as a tool that can facilitate the development and improvement of certain skills, such as digital communication, foreign language skills and information literacy. At the same time, students perceive that it is not able to facilitate to the same extent the development and improvement of skills such as critical thinking, creativity and face-to-face communication skills, which it seems can be attributed to ChatGPT's lack of human interaction and emotional intelligence. Students, in general, agree that ChatGPT is able to improve certain outcomes, such as student satisfaction, increase chances of completing the study and co-curriculum activities, compared to reducing under skilling, resolving skill obsolescence or improving students’ employability where students perceive ChatGPT as having a lesser impact. On average, public administration students also perceive ChatGPT as having a wide range of capabilities and, in the context of certain study issues, wider impact (Figs. 1 and 2) compared to ChatGPT abilities to facilitate skill development or improve outcomes, where students on average perceive these abilities as not so wide (Figs. 3 and 4). These findings have several practical implications within the higher education landscape. First, given that students perceive ChatGPT as a tool that can enhance learning experiences and help with their studies, higher education institutions should consider integrating ChatGPT into curricular and co-curricular activities. Second, while ChatGPT is seen as a valuable tool for enhancing digital communication, foreign language skills, and information literacy, higher education institutions should also recognize its limitations in fostering skills such as critical thinking, creativity, and face-to-face communication. Finally, since ChatGPT is also perceived as a tool to improve student satisfaction, higher education institutions should leverage ChatGPT to create a more engaging and efficient learning environment. These practical implications suggest that a balanced approach, combining artificial intelligence tools like ChatGPT with traditional teaching methods, is crucial for the holistic development of higher education. Nevertheless, further research is possible in the field of the reasoning behind the public administration students’ perceptions of ChatGPT capabilities and impact.