1 Introduction

As an information- and technology-centric society has emerged, interest grows in smart class environments that can make learners feel interested in learning processes, through newly increasing state-of-the-art technologies, and also enhance learning functions. The “Future of education in 2020 report; creation of future learning,” announced by the Education Future Foundation in the USA, forecasts that due to changes in the economy or society, traditional education should have become extinct in educational environments by 2020 [1]. This report forecasts that through changes in education, instead of students passively receiving information, in currently existing ways, active forms of education in which education is open and immediately connected on the spot will be realized, and the characteristics of collaboration, integration, and collective intelligence will be the center of education [2].

As digital convergence has been diffused due to the enhancement of digital technologies, wired Internet-based technologies and services have been replaced by wireless Internet-based devices and services. In particular, since 2009, as smartphones have become popular, the number of smartphone users has been rapidly increasing [3, 4]. This increase in smartphone use has brought about not only social changes but also changes in education. Diverse forms of applications for education implemented through smartphones have been already developed and utilized. According to the results of a survey conducted by the National Information Strategy Committee, a majority (51 %) of smartphone users answered that their information sharing activities increased because they were using smartphones, and a majority (51 %) of smartphone users perceived that smartphone use was effective in enhancing study or work productivity [5]. The survey results indicate that eventually, smartphones are very highly likely to be perceived and utilized as tools for the promotion of interaction or communication between users and for the enhancement of learning performance through teaching–learning in the field of education.

Centering on the government, educational studies are being conducted that aim to enhance the qualitative level of education, increase the effectiveness and efficiency of learning, and improve the continuity and sustainability of education through utilizing teaching–learning media into which cutting-edge information and communication technologies (ICTs) have been introduced. Given that a smart class includes diverse new functions, learners’ clear understanding of how to operate those environments or functions that have never been experienced will make usage easy and will ultimately lead to effective education. Norman stated that two factors, visibility and affordability, were core prerequisites for user-oriented software design [6]. That is, whereas poorly designed software cannot easily be used and does not provide sufficient clues for use, well designed software has visible functions and provides sufficient clues for appropriate utilization. Therefore, in order to design a user-oriented smart class, examining which elements trigger or support learning activities from the viewpoint of the main agents of learning—learners themselves—is more important than anything else. A more effective smart class can be designed in cases where the core factors for learning can be discovered from the viewpoint of learners.

The rest of this paper is organized as follows. In Sect. 2, the background of the study is introduced, and the ITLA system developed for the study is briefly introduced. In Sect. 3, experimental methods are explained, in Sect. 4, analysis results are explained, and finally, in Sect. 5, conclusions and future work are addressed.

2 Background

2.1 Smart class

Today, youth grow together with digital devices from the time of their birth, and they communicate and conduct their leisure activities in ways different from those used by earlier generations. These young people are called digital natives [7, 8]. In line with these learners’ needs and social and cultural changes, paradigms in educational environments are being changed into new educational paradigms. With these changes, state-of-the-art information technology (IT) is naturally infiltrating into the education fields. Open online educational environments such as open course ware (OCW) and massive open online course (MOOC) have already been activated, and many services are provided free of charge so that not only digital natives but also digital immigrants can be educated in new educational environments.

Many countries in the world have already been promoting mid-/long-term projects to prepare new education systems that will lead the changing sociocultural paradigms. In the Organisation for Economic Co-operation and Development, the Centre for Educational Research and Innovation implemented the “Schooling for Tomorrow” project. In this project, scenarios for the roles and visions of schools in future society were made, and ways in which school education should be changed and developed to be prepared for the future were presented through the development of tools to analyze trends. The Ministry of Education of the UK, through a project of the Futurelab named “Beyond Current Horizons,” has analyzed what changes will occur with respect to social and technical aspects of society during the next 20–30 years and how these changes will affect education. The Ministry of Education of the USA supported the Center for Technology in Learning at SRI International in creating a website named “School 2.0,” and it uses this website together with schools, regions, and general society to explore how future technologies will support education in a changed smart class [9].

These educational environments are being changed throughout the world, and to follow the global flow, Korean government announced and is implementing “a strategy to promote smart education” to construct smart class environments [10]. Using a smart class does not simply mean using state-of-the-art digital devices, but “smart class” has a comprehensive meaning that enables learning motivation, sharing, interactions, collaboration, and self-directed learning through state-of-the-art ICTs. In Korea, this is called “smart education” and has been diversely defined by many researchers.

The Ministry of Education, Science and Technology stated that smart education is an intelligent customized learning system for strengthening learners’ capability in the twentyfirst century that means the motive power to innovate education systems such as educational environments, content of education, education methods, and assessment [10]. Lim emphasized the term, “smart education” to mean such things as e-Learning or mobile learning that emphasizes the teacher, learner, and content-oriented educational informatization policies [7]. Table 1 shows the diverse definitions of “smart education” by researchers.

Table 1 Definition of “smart education”

Smart class markets in Korea are expected to reach 3.5 trillion won ($3 billion) by 2015, and related industries such as software, hardware, content, and e-book industries are expected to be further activated [14]. With advantages such as state-of-the-art ICTs, intelligent systems, digital textbooks, and content, a smart class can form abundant educational environments, enhance the rate of students’ participation in learning by maximizing the effects of education, and enable teachers’ efficient teaching activities. However, the limitations of a smart class must be overcome. There are arguments that state-of-the-art technologies may rather disturb learning activities and problems occurred in a smart class such as a decline in concentration, low readability of digital textbooks, and distracting class atmosphere must be resolved. In fact, in e-learning education, there have been cases where as cost effectiveness was emphasized, strategies to promote learners’ acceptance and utilization were overlooked, leading to relatively low satisfaction compared to the rate of participation in education. For new education methods such as a smart class to be accepted in the field of education, studies of factors that bring about learners’ satisfaction cannot but be perceived as important [15].

Therefore, to enhance students’ intentions to accept a smart class and enhance their satisfaction with it, factors that cause their resistance to learning environments or those that can enhance their satisfaction should be systematically studied. This study is to explore factors that affect learners’ satisfaction within a smart class and then to support decisions in the design of smart classes. The purpose of the study was to answer the following questions:

RQ1:

Does the system playfulness affect satisfaction of the smart class?

RQ2:

Does the interaction affect satisfaction of the smart class?

RQ3:

Does the attitude toward smart devices affect satisfaction of the smart class?

RQ4:

Does the attitude toward other classes affect the satisfaction of the smart class?

To that end, first, an integrated teaching and learning assistance (ITLA) system was designed and developed to construct a smart class environment, and the constructed environment was applied to an elementary school for 4 weeks. Then those factors that affected satisfaction with the smart class were analyzed, and among them, the factors that had the largest and the smallest effects were examined to effectively support decisions in smart class design.

In this study, we define a smart class as the class of facilitating collaborations, communications, and participation using the ITLA system. Figure 1 shows the smart class model. The smart class is equipped with electronic whiteboard, electronic lecture desk, smart devices, and a wifi environment. Instructors and students can use various services (digital textbook, learning contents, drawing tool, monitoring system, etc.) from the ITLA system through the smart class, which facilitates collaboration, communication, and participation within their classroom.

Fig. 1
figure 1

Smart class model

2.2 ITLA system

A change in the IT environment has resulted in the transferring of all educational materials into multimedia form, the optimized education solution for the new educational environment. The demands on the utilization of ICT for education are also increasing in order to eliminate the unilateral and standardized delivery of knowledge via textbook [16]. The education paradigm is changing into the form of a cooperative, sharing, participative, and customized education. So an optimized integrated solution is necessary in order to create a smart class for the new educational environment. In this study, a structured, plug-in ITLA system was applied to the smart class, which can improve the effects of education by maximizing cooperative, sharing, and participative learning, reflecting the characteristics of the learning environment and digital natives as learners. Teachers and students can participate in lectures through effective personalized and customized education [17].

2.2.1 Service architecture

ITLA systems provide teachers and students with new service-based education processes in a package that can provide education that enables collaboration, sharing, and participation. Through ITLA systems, classrooms that enable collaboration, sharing, and participation can be implemented using diverse learning support tools in teachers’ and students’ smart devices, and the systems will provide personalized and adaptive education content through profiling the individual students.

One major service scenario is as follows. The content provider uploads teaching materials such as digital textbooks, videos, and PPTs onto the content management system. Teachers create classes and curricula and can make their own digital teaching materials utilizing registered educational content. In classrooms, the teachers can implement their classes using electronic backboards and digital teaching materials made on smart devices, and students can study during class hours and after class hours through digital teaching materials.

All the learning results are stored in digital teaching materials and class teaching materials, and educational content that fits the levels of the students can be automatically recommended based on the stored data. In addition, parents can monitor their children’s learning information.

Figure 2 shows a model scenario for classes using ITLA systems.

Fig. 2
figure 2

Scenario of lesson model

E-portfolio systems (EPS) are divided into those for teachers and those for students, and teachers can select diverse tools such as real-time learning monitoring system (RTS), a mindmap collaboration system (OKMMS), a smart digital textbook creating system (SBMS), and a drawing tool system (DTS) on electronic whiteboards and smart pads. Students can also select classes to use diverse tools such as OKMMS, SBMS, and DTS on their smart pads. Teachers can access the ITLA system website to easily create digital teaching materials, and teachers and learners can access the ITLA system website to easily use the diverse tools. An app version for smart pad access was also developed.

2.2.2 Application architecture

The ITLA system supports diverse systems such as DTS, RTS, intelligent tutoring system, smart digital textbook creation system, and content retrieval and a semantic search engine in the form of a plug-in [17]. Figure 3 shows the entire application architecture.

Fig. 3
figure 3

Structure of structured plug-in ITLA system

The ITLA system is a type of plug-in that can be built into various systems upon user request. E-portfolio system (EPS) and an integrated plug-in system (IPS) are essential systems that consist of a base system (ILE). Other systems are divided into those that are essential for EPS and those that can independently plug in. This environment was designed and developed to consider expandability, to pursue open systems rather than closed systems, and to enable the organic plug-in of individual systems. The form of plug-in was created so that it could provide systems according to user requirements. It combines necessary systems in modules that satisfy the requirements of the government, enterprises, or schools that wish to provide smart class services to enable the implementation of adaptive systems.

ITLA systems are integrated education content services that enable the linkage of learning content from management and production to teaching–learning, based on international standards. The implemented ITLA system enables content management and production based on the curriculum system in Korea and supports automated content conversion and transmission technologies that automatically convert content formats according to the characteristics and services of the diverse smart devices. School and home can be formed as one learning space for learners through the intelligent tutoring system and the EPS to provide strong learning associations.

3 Experiment methods

3.1 Questionnaire design

To prove research questions with the smart class applied with the developed ITLA system, a smart class satisfaction questionnaire was developed. A total of 219 people participated in the questionnaire survey, resulting in questionnaire results for 210 participants, excluding the inadequate questionnaire results of nine participants. Inadequate questionnaire results included those questionnaire sheets in which all responses received the same number or in which any of the questions had not been answered.

The smart class satisfaction questionnaire was divided into the categories, digital friendliness, attitude toward smart devices, Interaction, ITLA system playfulness, perceived usefulness, perceived ease of use, attitude toward class, and smart class satisfaction. The composition of the measurement variables used in the present study and the operational definitions of individual factors are as shown in Table 2 below.

Table 2 Operational definitions of individual factors and related studies

The questionnaire used in the study was developed with a total of 33 questions. Before the questionnaire survey was conducted, in order to assess the validity of the developed questionnaire questions, an expert group was organized to conduct content validity ratio (CVR) tests. The CVR test is one of the test methods used in academic and vocational testing. To conduct this test, expertise in the given topic is required. This test was developed by C. H. Lawshe [36, 37]. The equation for CVR calculation formula is Eq. (1) [38] (Table 3).

$${\text{Content}}\;{\text{validity}}\;{\text{ratio (CVR)}} = \frac{{n_{e} - {\raise0.5ex\hbox{$\scriptstyle N$} \kern-0.1em/\kern-0.15em \lower0.25ex\hbox{$\scriptstyle 2$}}}}{{{\raise0.5ex\hbox{$\scriptstyle N$} \kern-0.1em/\kern-0.15em \lower0.25ex\hbox{$\scriptstyle 2$}}}}$$
(1)

n e the number of respondents who checked “important” or “very important”, N the total number of respondents

Table 3 Minimum CVR value based on total number of respondents

To test the validity of the questionnaire developed for this research, 10 professional testers participated in the verification process. For expert recruitment, a recruitment letter was sent to experts who satisfy the criteria below. Then experts who want to participate in our study for CVR testing received information about the CVR and our study. The professional testers who participated in this test who satisfy the following criteria full filling at least one requirement:

  1. 1.

    Have more than 5 years career experience

  2. 2.

    Have a doctorate degree or an authorized instructor

  3. 3.

    Have a minimum of 3 years career experience in computer-related lectures

  4. 4.

    Have experience in carrying out research on smart education

The average, minimum, and maximum CVR test results were .75, .2, and 1, respectively. Accordingly, the item for which the CVR was <.62 was modified or deleted according to the feedback from professionals and teachers. In order for the questionnaire to analyze satisfaction with the smart class, 26 items constituting 8 categories and a 5-point Likert Scale were developed as well. Table 4 shows the final questions of the developed questionnaire.

Table 4 Questions of questionnaire

In the case of confidence verification, when the same concept has been repeatedly remeasured through tests of validity of measurement items, if similar results are shown without being affected by the situation or time, the results can be regarded as reliable and consistent. To measure the confidence of the questionnaire developed in the study, the confidence was verified through Cronbach’s alpha coefficients used to determine whether individual questions had internal consistency. The confidence was shown to be high, as Cronbach’s alpha coefficients for all latent variables were at least .8. The factor analysis is an analytic technique that explains the relationships between variables using latent common factors between mutually correlated variables. It finds and analyzes latent common factors that can explain relationships between variables that have not been observed, utilizing statistical models that show the relationships between these variables and latent common factors. According to the results of the factor analysis of the developed questionnaire, the factor loadings of all measurement variables were at least .5, indicating that these variables are important. Finally, the relationships between common factors by area were statistically examined through confidence measurement through Cronbach’s alpha coefficients and exploratory factor analyses. Table 5 shows the Cronbach’s alpha values and the results of the exploratory factor analyses.

Table 5 Exploratory factor analysis and resultant Cronbach’s alpha values

3.2 Class design

In order to survey, the classes were conducted for approximately 4 weeks, from October 14 to November 6, 2013. The classes were applied only to social study class hours, and the classes were done with two units of the Korean social study curriculum. The classes were done three times per week, and each class hour consisted of instructions for 45 min and a rest for 10 min.

For class, one electronic whiteboard, one electronic lecture desk, smart pads (20 iPads, 20 Galaxy pads), and wireless sharers were installed in the science room to construct a smart class environment. The teacher and students of each class moved to the science room in which the smart class environment was constructed when each social studies class was implemented (Fig 4).

Fig. 4
figure 4

Constructed smart classroom

During the application period, a class operation report was prepared for each class. The class operation report contained the preparer, date and time, teacher name, place, user, content of operation, requirements, defects, and issues. Before the test application of the smart class, teacher training was done for 1 week in the smart class environment, to allow the teachers’ optimum class operation.

When the classes were implemented, each of the students was provided with one smart pad to use during class hours. The electronic whiteboard and electronic lecture desk were used by the teachers during class hours.

The students could access the ITLA system during class hours using the ID and password for access provided to each of them. Thereafter, in the ITLA system, they could see class teaching materials and use events such as quizzes, drawing tools, questionnaire surveys, and mind maps. After students’ events, the results of the events could be identified in real time on the electronic whiteboard, and anybody could access the ITLA system to identify the results of the events even when the class had finished. Teachers could perform diverse activities in classes utilizing the ITLA system and could form powerful learning environments for learners. Learners could learn by themselves, sometimes through the ITLA system, and teachers could play the role of helpers to help the progression of the students’ learning, because the ITLA system provided not only digital textbooks but also an intelligent tutoring system.

3.3 Participants

A smart class with the ITLA system was applied to six classes of fourth graders in an elementary school located in Gyeonggi-do, south Korea. The number of students per class was approximately 37, and a total of 219 students participated in the study. The participants consist of 117 male students (53.42 %) and 102 female students (46.58 %). With one teacher per class, a total of six teachers participated in the study. The following table shows the number of participants (Table 6).

Table 6 Participants

3.4 Operation report

In additional, to collect information about the teachers’ requirements and the system defects and issues, and to reflect the results to the system, an operation report was prepared for every class. These operation reports for all classes were integrated and documented once a day. Twelve operation reports were prepared. Table 7 shows an operation report for three classes in which classes were implemented from 9:00 to 14:00 on October 22, 2013.

Table 7 Operation report for October 22, 2013

While the model school was implemented, requirements, defects, and issues were collected, and the results were reflected on the ITLA system.

4 Results and analysis

After test applying the smart class for 4 weeks, a questionnaire survey was conducted on 219 participants, and an analysis was conducted based on questionnaire sheets from 210 participants, excluding nine questionnaires with insincere responses. Table 8 shows the numbers of samples, means, standard deviations, skewness coefficients, and kurtosis values by questionnaire item.

Table 8 Descriptive statistics of individual measurement items for variables

Based on the results of descriptive statistics of the questionnaire items in the study, there was no case where the standard deviation was larger than 3, the absolute value of skewness was larger than 3, or the absolute value of kurtosis was larger than 8. It implies that the results of students’ responses to the questionnaire can be regarded as having normal distributions.

According to the information recommended by the American Psychological Society for goodness of fit of information measurement models, smaller χ2 values are more desirable, and P values (≥.05) and root mean-square residuals (RMR) for χ2 values smaller than .05 and closer to 0 are better [39]. Models with .9 or higher normal fit index (NFI), .9 or higher Tunker-Lewis index (TLI), and comparative fit index (CFI ≥ .9) closer to one can be judged as better models [40].

The goodness of fit indexes for the factors set in the present study were χ2 = 342.449 (df = 271, p = .000; p ≥ .05 is desirable), χ2/the degree of freedom (Q value) = 1.264 (≤3 is desirable), GFI = .892 (≥.90 is desirable), AGFI = .860 (≥.80 is desirable), NFI = .878 (≥.90 is desirable), and RMSEA = .036 (≤.05 is desirable). Although the p value for χ2 was lower than the threshold, other goodness of fit indexes were shown to be at satisfactory levels, and thus the factors were judged as not hindering single dimensionality. In addition, all the composite reliability (CR) and average variance extracted (AVE) values for individual study variables were shown to be higher than their general thresholds of .7 and .5, respectively. Therefore, the measurement items can be said to be sufficiently reliable and to have representability for the relevant variables. Table 9 shows the results of the verification of goodness of fit.

Table 9 Results of verification of goodness of fit of confirmatory factor analysis

Validity is a concept related to the degree to which constructs to be measured have been accurately measured, and it is divided into content validity, criterion-related validity, and construct validity. Among these, construct validity means judgment regarding whether the abstract constructs to be measured are actually measured by the measuring tools. To gain recognition of construct validity, both convergent validity and discriminant validity should be presented; methods of enabling these include multi-trait-multi-method matrixes and factor analysis.

All the factor loadings in the study have statistically significant t values. Therefore, the convergent validity and discriminant validity among individual constructs were secured. In addition, all CR and AVE values for individual study variables were shown to be higher than general threshold values (CR: .7, AVE: .5) in all items except for the DF area. Therefore, the measurement items can be said to be reliable and to have representability for the relevant variables. In addition, if the AVE value obtained between two factors is larger than the coefficient of determination (r2) of each factor, it can generally be said that the discriminant validity has been secured between the two factors.

Variables that show the highest correlation in Table 10 are ITLA system playfulness and smart class satisfaction, and their correlation coefficient is .659, of which the square value is .434. Among the values presented in Table 10, all AVE values except for that of DF are higher than that, indicating that the discriminant validity has been secured. When put together, the reliability and validity of the variables used in the present study can be said to have been secured.

Table 10 Table of analysis of discriminant validity between the constructs

In the study, factors that affect satisfaction with the smart class’s learning environments were analyzed through structural equation models. Structural equation modeling (SEM) is a technique for the analysis of causality among diverse latent variables through measurement models and structural models that apply confirmatory factor analysis (CFA) in order to find latent variables without any measurement error and to analyze theoretical causality among latent variables that cannot be directly measured, connecting individual factors according to the causality [41]. To analyze causality between diverse latent variables, SEM analysis was conducted by applying CFA. Table 11 shows the results of verification of the hypotheses by the SEM analysis.

Table 11 Results of verification of the hypotheses by the SEM analysis

The important learning factors that affect smart class satisfaction found in the study include four factors; ITLA system playfulness, perceived usefulness, perceived ease of use, and attitude toward class. Of these, perceived ease of use did not directly affect smart class satisfaction but did affect attitude toward class through perceived usefulness. Based on result of study, research questions were proved as follow:

RQ1

“Does the system playfulness affect satisfaction of the smart class?” has been proved

RQ2

“Does the interaction affect satisfaction of the smart class?” has been disproved

RQ3

“Does the attitude toward smart devices affect satisfaction of the smart class?” has been disproved

RQ4:

“Does the attitude toward other classes affect the satisfaction of the smart class?” has been proved

Among the factors of ITLA Systems, the factor interactions that enable multidirectional communications between teachers and learners and between learners have been considered very important in estimating the effectiveness of learning [42]. However, there have been contradictory opinions including study results indicating that the interactions may lead to anxiety due to the lack of offline interactions and a sense of alienation [43, 44]. Interactions are thought not to have shown significant associations with smart class satisfaction in the study, due to the fact that experimental subjects were in lower grades in elementary school, and the negative perceptions of interactions resulting from the negative aspects of problems such as communication outcasts are interlocked with the recent rapid spread of smartphones in Korea.

The reason why attitude toward smart devices showed a low path coefficient to perceived usefulness and why perceived ease of use showed a low path coefficient to attitude toward class is that domestic elementary school students in recent times have grown up experiencing diverse digital devices since they were young. Youths of the present time, the so-called “digital natives,” should have felt little anxiety or inconvenience in relation to the use of the ITLA System, unlike adults who feel a fear of new IT devices. These students experienced the ITLA System when they were already highly familiar with digital devices.

ITLA system playfulness showed a high path coefficient of .640 to smart class satisfaction, indicating that the playful factors of the ITLA System inspired great interest in the students. Therefore, in relation to the development of integrated solutions for the application of a smart class, playfulness factors are considered to greatly affect students’ satisfaction.

5 Conclusions

Because of the pluralism and uncertainty of future society, demands for new education that can raise learning capacity in the twenty-first century in step with the stream of social changes are continuously increasing. As if reflecting this, many countries in the world have already been preparing future types of schools in line with changing educational environments. Therefore, the elements that go into the effective design and application of a smart class should be examined, because if a smart class is constructed and applied without any consideration, the risk of having it go in a wrong direction without obtaining any positive educational effect will be high.

In this study, the elements that should be considered for the effective design and application of a smart class were examined. For the analysis, an ITLA system necessary for smart class environments was developed and test operated for 4 weeks in an elementary school. According to the results of the analysis, Playfulness of the ITLA System was the factor that had the largest effect. This does not simply mean only playfulness, but it can refer to the funny and interesting elements of those educational solutions (software) necessary for an effective smart class that are used firsthand and that are utilized by teachers and learners. In fact, current development of the smart class in Korea is mostly focused on the construction of infrastructure and hardware, and the development of integrated solutions to be used by teachers and learners is insufficient. Although the factors digital friendliness and attitude toward smart devices were expected to greatly affect the smart class, contrary to expectations, those were found through analysis to be factors that did not have great effects on the smart class. Given this fact, the experimental subjects (fourth graders in elementary school), who fall into the category of digital natives, can be regarded as having no anxiety or inconvenience related to the use of digital devices, because unlike adults, they are already familiar with these devices. As for the factor, attitude toward smart devices, it can be said that smart devices per se cannot be regarded as a factor in obtaining educational effects in classes, because smart devices are only instrumental hardware. The class strategies presented by teachers via smart devices and whether effective class strategies that could not be used earlier can now be used because smart devices can be utilized are among the factors for obtaining significant educational effects in a smart class. Perceived usefulness and perceived ease of use can be regarded as very important elements in the product development of software development technology fields. Since a smart class is also intended to provide positive educational effects based on the development and educational utilization of state-of-the-art technologies, the elements perceived usefulness and perceived ease of use were found to importantly affect the smart class, as expected.

Through the study, factors (ITLA system playfulness, perceived usefulness, perceived ease of use, attitude toward class) that must be considered in the design and application of an effective smart class can be suggested to educators, researchers, developers, and education policy decision makers to support their effective decisions.

In addition, for future studies, model schools should have smart classes applied for 1 or 2 years in order to collect more reliable data through long-term model schools, and the diverse factors that must be considered in a smart class must be analyzed through scientific analytic technologies.