Keywords

1 Introduction

Since 2001, China issued the outline for the development of educational informatization in the Tenth Five Year Plan and fully implemented the “school to school” project, a great wave of educational informatization construction has been set off all over the country, and substantial progress has been made in funding, construction scale, software and hardware platform, application promotion and other aspects [1]. China’s education information construction is entering a period of rapid development. In the process of educational information construction, our government has always paid close attention to educational information as a government behavior, as an important strategy to promote educational reform and development. China’s education informatization is changing from construction to application, and “from construction to application, to promote development by application” is the theme of China’s education informatization at this stage. To promote development by application, it is necessary to drive application by performance evaluation. In the aspect of performance evaluation of educational informatization, both at home and abroad are in the exploratory stage.

The performance evaluation of education informatization is a relatively difficult matter. On the one hand, because education informatization is not only a dynamic development process, but also a problem of multiple inputs and multiple outputs, its output is not easy to measure with quantitative indicators. On the other hand, there is no mature theoretical guidance and suitable measurement methods and measurement tools in this field, coupled with the rapid changes in the field of education informatization, so the performance evaluation of education informatization has become a very important and considerable concern for everyone. As a combination of teaching and technology, multimedia teaching has been recognized by the teaching community for its vivid teaching style, colorful teaching content, and scientific and comprehensive teaching strategies. As the main tool of multimedia teaching, multimedia courseware has become the key to affect the effect of classroom teaching. Under the dual guidance of social demand and education policy, multimedia teaching emerges as the times require, and has been widely used and recognized by all kinds of schools. However, in the process of actually using multimedia teaching, the advantages of multimedia technology have been brought into play due to the influence of traditional teaching concepts, teachers’ educational technology capabilities, and multimedia courseware making technology. Multimedia courseware, with its powerful functions, has stimulated people’s passion for knowledge. Its rich content, vivid pictures, clever link combinations, colorful presentation effects, and its ease of learning and ease of use have quickly become indispensable tools for the development of modern education. Multimedia courseware is the main form of multimedia teaching. The quality of multimedia courseware production directly affects the quality of multimedia teaching. First of all, we should make clear the concept of multimedia. In the field of computer, multimedia refers to a variety of media that represent information, usually including: text, graphics, images, audio, video, animation, etc. The so-called multimedia technology is to use computer technology to carry out interactive comprehensive processing and control of text, graphics, voice, image and other information media, establish logical relations, and integrate them into a system with interactivity. Multimedia courseware is a CAI teaching software designed and developed by using multimedia technology. In terms of implementation technology, multimedia courseware is a kind of multimedia software that uses multimedia technology to deal with information media such as text, picture, sound and image interactively and comprehensively to express teaching content. In terms of courseware content, it is guided by teaching theory and learning theory, uses the method of system theory, and reasonably selects and designs teaching information according to the characteristics of teaching objectives and teaching objects Media and organic combination to form an optimized teaching structure of a teaching system. However, in the actual classroom teaching, with the increase of the utilization rate of multimedia courseware, there are also many problems. In order to understand the actual situation of multimedia courseware teaching and truly grasp the current status of the effectiveness of multimedia courseware in classroom teaching applications, based on this, put forward Research on Evaluation Method of Multimedia Art Teaching Course Playing Effect Based on Data Envelopment Analysis.

2 Evaluation Method of Multimedia Art Teaching Courseware Playback Effect Based on Data Envelopment Analysis

Data envelopment analysis (DEA) solves the above problems. It is an efficiency evaluation method used to evaluate the relative effectiveness (called DEA effectiveness) between the same departments. It was established by a. charnes and w.w. cooper in 1978 and can be widely used in performance evaluation.

2.1 Determine the Evaluation Subject

This article uses a combination of self-evaluation and other evaluations when evaluating teaching effects. The evaluation of the teaching effect is carried out from two aspects, one is the evaluation of the teachers in the class, and the other is the evaluation of the students [4]. The evaluation factors of teaching effect are divided into two parts: one is the evaluation factor set of teachers in the curriculum, which is used for the evaluation of teachers by students and tutors; the other is the evaluation of students’ peers (including their own evaluation). Because there are many evaluation factors, the principal component analysis method can be used to find the main factors that are not related, as shown in the Fig. 1.

Fig. 1.
figure 1

Set of factors for evaluating teaching effects

Full size image

The evaluation of the whole teaching effect is divided into three parts, so the main body of the evaluation is composed of students, teachers and leaders in charge.

2.2 Determine the Evaluation Index and Its Weight

Generally speaking, the importance of each indicator is different. In order to reflect the importance of each indicator, Assign a weighting factor \( a_{i} \) to each indicator \( f_{i} \) and satisfy \( a_{1} + a_{2} + \ldots a_{n} = 1 \) [5]. The determination of the weight set is an important step, it will directly affect the final result. Generally, the collective experience judgment method, expert consultation method, and analytic hierarchy process can be adopted, and different methods can be selected according to specific situations.

First of all, data envelopment analysis is used to measure the past situation of each evaluation object, and the index obtained can reflect the strength of the evaluation object to a certain extent, as its initial condition, this index is called the reference index; the current situation is measured by the same method, and the index obtained is called the current index [6]. The array of the reference index and the current index is called the index state of the measurement object. Let \( x_{j} \) be the reference index of the \( j \) th evaluation object, and \( y_{j} \) be the current index of the evaluation object, where \( x_{j} \in E_{1} \), the array \( \left( {x_{j} ,y_{j} } \right) \) is called the index status of the \( j \) th evaluation object.

Let us observe that the index state of each evaluation unit is \( \left( {x_{j} ,y_{j} } \right),\,j = 0,1,2, \ldots \ldots n \), and:

$$ T = \left\{ {\left( {x,y} \right)\sum\limits_{j = 0}^{n} {\lambda_{j} y_{j} \le y,} \sum\limits_{j = 0}^{n} {\lambda_{j} \ge 0,\,j = 0,1,2, \ldots \ldots n} } \right\} $$
(1)

The exponential state set consisting of the above exponent states \( \left( {x_{j} ,y_{j} } \right),\,j = 0,1,2, \ldots \ldots n \), where \( \left( {x_{0} ,y_{0} } \right) = \left( {0,0} \right) \) is. The exponential state set \( T \) is obviously a convex set, that is, if \( \left( {x^{n} ,y^{n} } \right) \in T \), then \( \left( {\lambda x^{\prime } + \left( {1 - \lambda } \right)x^{n} ,\lambda y^{\prime } + \left( {1 - \lambda } \right)y^{\prime \prime } } \right) \in T \). Among them, \( 0 \le \lambda \le 1 \).

From this, we can establish the above possible set of exponential states.

After the indicators are obtained, the relevant weights can be calculated according to the rating of the comments and their membership vectors, which are:

$$ V = \left\{ {V_{1} ,V_{2} ,V_{3} ,V_{4} } \right\} $$
(2)
$$ E = \left( {0.9,0.7,0.5,0.3} \right) $$
(3)

In the formula: \( V \) represents the level of student’s cognitive ability, that is, the teacher’s comments on the students, which are represented by good, good, average, and poor respectively; \( E \) represents the attribute vector of the evaluation index status

2.3 Establish Evaluation Matrix

Firstly, a questionnaire [7] is designed, which includes the evaluation form for students and teachers in charge, and the evaluation form for students (including mutual evaluation between students). The evaluation of students is mainly the evaluation of their recent learning., Is a process of vertical comparison [8], analyzing its recent improvement through learning. With the further development of multimedia courseware teaching, it is not difficult to see that many unreasonable uses have deviated from the purpose of multimedia courseware teaching, entered the misunderstanding, and failed to play its unique functions. The specific contents of the questionnaire analysis are as follows:

  1. (1)

    Technical evaluation. Technical evaluation includes five aspects: interface design, courseware interaction, opening and expansion, media auxiliary function, and material quality. Except for media material, students are not satisfied with other technical evaluation indexes of multimedia courseware evaluation;

  2. (2)

    Scientific level. Scientific evaluation includes five aspects: scientific advancement, scientific rationality, text and graphics, scientific content and scientific design. Students are not very satisfied with the three aspects of scientific advancement, scientific rationality, text and graphics. It can be seen that the multimedia courseware of colleges and universities lacks the advanced nature first, that is, it cannot reflect the advanced results of introducing advanced scientific research and teaching research at home and abroad;

  3. (3)

    Usability evaluation satisfaction. Usability evaluation includes user guidance, operation and use, software operation and software performance. Students are not satisfied with the first two, that is, there are no effective user guide manual, computer operation interface, operation steps and convenience;

  4. (4)

    Educational evaluation and artistic evaluation. Educational evaluation includes: teaching adaptability, structural rationality, cognitive regularity, vivid interest, and evaluation feedback. Artistic evaluation includes audio and video materials, reading volume, dubbing effects, and style consistency.

Then the two tables are summarized according to each evaluation factor, and the evaluation matrix is as follows:

$$ R_{ij} = \frac{{N_{ij} }}{N} $$
(4)

Where \( N_{ij} \) is the number of comments \( V_{j} \) obtained by the \( i \) index \( f_{i} \), and \( N \) is the number of people. Each row of the matrix adds up to 1.

2.4 Constructing a Reasonable and Scientific Evaluation Index System of Multimedia Courseware

In accordance with the principle of constructing multimedia courseware evaluation index system, the five dimensions of the summarized courseware evaluation are specifically detailed, and then constitute the following multimedia courseware evaluation index system.

Here we use the evaluation score interval variation division. The evaluation level is divided into five levels. We make the following statements for these five levels.

Level 1: indicates that the teaching software not only has the basic characteristics of teaching software, but also has considerable flexibility and intelligence in all aspects. It provides learners with very reasonable and friendly learning support and learning mechanisms, which can effectively achieve high The learning effect and help to promote student creativity and truly realize the educational advantages of computers. The comprehensive comments given by such teaching software should be: excellent, worthy of promotion, and analysis, to promote the development of computer-aided teaching.

Level 2: it means that the teaching software has the expected basic characteristics of the teaching software, and has quite strong flexibility in some aspects. At the same time, it has the unique features that other teaching software does not have, and has a breakthrough effect. It should be a relatively good teaching software.

Level 3: It indicates that the teaching software has the basic production goals of the desired teaching software, such as high interface presentation quality, good teaching software reliability, and teaching software that can overcome the teaching difficulties of ordinary teaching methods, but this teaching software lacks the corresponding flexibility., There is no particularly obvious unique point, the comprehensive comment given by such teaching software should be: acceptable and can be promoted in a small range, while pointing out the defects of this teaching software, with a view to further improvement.

Level 4: indicates that the teaching software deviates from the ideal state it should achieve. For example, the learning mechanism provided needs to be further explored, the quality of information presentation needs to be further improved, no personalized teaching measures have been taken, and every link of software development has not been in place, but the software itself has its merits. The comprehensive conclusion given by such teaching software should be: put forward This paper puts forward the quality objectives and teaching objectives that should be achieved by the teaching software after the change. At present, this software is not acceptable, and it is the most evaluated after the change.

In the implementation of courseware effect evaluation, we should pay attention to three evaluation elements, the evaluation subject (evaluator), the evaluation type, and the evaluation object (teaching software). We divide evaluators into four categories: teaching experts, technical experts, practical experts, and learners. These four types of evaluators each have their own expertise, so different types of evaluators will have different evaluation authority for different evaluation types, so different types of evaluators should have different weights in different evaluation types. . A teaching expert is often proficient in all aspects of teaching. He can scientifically evaluate the effectiveness of the teaching function of teaching software and give a more scientific score. The technical expert is mainly familiar with computer technology, and he will give a more reasonable score in technical evaluation than other evaluators. Practice experts are familiar with the entire process of software compilation, and have also done research in education and computer technology. Therefore, practice experts have a more even distribution of weights in process evaluation, technology evaluation, and education function evaluation. According to age, knowledge background, learning environment and the times of contacting Cai, the evaluation of teaching software will produce different evaluation results. For example, a learner who has never been in contact with a computer uses the teaching software for the first time. When learning, there may be a sense of curiosity, so their evaluations are higher, so learners’ three different types of evaluation in teaching software are often unreliable. We do not require them to conduct a systematic evaluation of teaching software, but they can reflect the learning effects produced by teaching software.

2.5 Comprehensively Determine the Grade and Sort

The calculation of the decision level first requires the use of formula \( S = W \cdot R \) to calculate the comprehensive evaluation vector [9]. Adjust the value so that the sum is 1.

Among them: \( W \) indicates the teacher’s teaching weight index, which is the highest value; \( R \) indicates the student’s awareness of participation throughout the class.

Calculate the comprehensive priority according to formula \( N = S \cdot E^{T} \) [10].

Among them, \( N \) represents the final judgment level, and \( T \) represents the number of students. Art teaching activities belong to art teaching, and the quality of teaching effect can not be simply reflected in the test paper, which has certain fuzziness. In this paper, the use of data envelopment analysis to evaluate the effect of multimedia art teaching for students can not only determine the level of teaching effect, but also quantitatively see the degree of belonging to the level, which has achieved good results in practical application.

3 Experiment and Analysis

In order to better see the effectiveness of the design method in this paper, it is specially compared with the traditional evaluation method.

3.1 Experiment Preparation

Based on the above theory, a new teaching evaluation method is developed, and an empirical study on this method is carried out. The data in this paper are all from the art teaching group report card of the basic teaching and Research Office of Shaanxi art college, including the average score of the final art examination of the first semester of 2009–2010 academic year and the average score of the art examination of the corresponding classes. Table 1 It is the original data table (the real name of the teacher has been hidden in the table), and it is sorted according to the current scores of the final examination of the first semester. Using the evaluation method of multimedia art teaching courseware playback effect based on DEA designed in this paper, the playback effect value of these 9 teachers’ multimedia art teaching courseware is calculated, which represents the evaluation index of teachers’ teaching effect, and the ranking of teachers’ teaching effect is listed in Table 2 according to the index.

Table 1. Current scores and ranking of classes taught by teachers
Full size table
Table 2. Teaching effect value and new ranking
Full size table

3.2 Result Analysis

According to the different results of the two tables, the evaluation method of multimedia art teaching courseware playback effect based on data envelopment analysis effectively eliminated the effect of the difference in initial conditions. Different, the ranking of 8 teachers has changed, making the evaluation of teaching management evaluation more objective and persuasive. In the example, there are fewer input indicators and output indicators, so the teaching effect value evaluation system obtained is relatively simple. In actual applications, the input and output indicators can be increased accordingly to obtain more real and reliable data.

In order to further verify the effectiveness of the method in this paper, the evaluation accuracy of the proposed evaluation method of the playing effect of multimedia art teaching courseware based on DEA is compared and analyzed with that of the traditional evaluation method of the playing effect of multimedia art teaching courseware based on fuzzy mathematics, and the comparison results are shown in Fig. 2.

Fig. 2.
figure 2

Comparison results of evaluation accuracy

Full size image

As can be seen from Fig. 2, the evaluation accuracy of DEA based evaluation method is as high as 74%, while that of fuzzy mathematics based evaluation method is as high as 34%, indicating that the evaluation effect of this method is good.

4 Concluding Remarks

In view of the poor evaluation results of the traditional multimedia art teaching courseware, this paper puts forward the evaluation method based on data envelopment analysis. With the help of different evaluation subjects’ visual perception of use, and with the help of the determination of comprehensive evaluation vector, the effect of courseware playing is evaluated one by one. Teaching evaluation is only a means, the purpose is for teachers to better improve the quality of teaching. Through the empirical analysis of the above example, this method can evaluate the teaching quality of each teacher more fairly and reasonably, and comprehensively consider the initial differences of students’ scores before entering school. The teacher can also find the differences and analyze the reasons from them, which is conducive to the better development of teaching management activities, the improvement of the evaluation accuracy of multimedia art teaching courseware playback effect, and the multimedia teaching Learning provides a theoretical basis.