Efficiency Assessment of Computer-Oriented Learning Environment of an Institute of Postgraduate Pedagogical Education: Factors, Criteria, Characteristics

Abstract

This article provides the rationalization for a comprehensive assessment of computer-oriented learning environment efficiency of an institute of postgraduate pedagogical education and provides the definitions of the ‘efficiency of computer-oriented learning environment of an institute of postgraduate pedagogical education’, and the ‘assessment criteria for computer-oriented learning environment efficiency of an institute of postgraduate pedagogical education’. In order to measure the efficiency of computer-oriented learning environment of an institute of postgraduate pedagogical education, a multi-criteria factor evaluation model was created. It includes, in the first place, the factors defining computer-oriented learning environment of an institute of postgraduate pedagogical education. The article provides the rationalization for and describes the developed criteria of efficiency assessment in such an environment and divides them into corresponding criterion characteristics as well as establishes the numeric values of coefficients of importance of all characteristics, criteria and efficiency factors of computer-oriented learning environment of an institute of postgraduate pedagogical education.

1 Introduction

The need for drastic changes required to improve the quality and competitive ability of education in the new economic and socio-cultural environment, the acceleration of Ukraine’s integration into the international education environment demands the introduction of modern information and communications technologies (ICT) that improve learning process (LP), the accessibility and efficiency of education [25].

The task of efficiency assessment of learning process, management decisions, teaching techniques, etc., the functioning of an educational institute in general, in terms of the use of information and communications technologies is partly accomplished during the procedure of licensing and accreditation. These procedures at higher educational establishments (HEE) are regulated by the fundamental documents on education policy in Ukraine, for example [4, 5, 19–21], etc. These documents require the conformity of performance characteristics of educational establishments to accreditation criteria.

Computer-oriented learning environment of an institute of postgraduate pedagogical education is a controlled and deliberately created artificial space intended for unfolding the learning process by the means of ICT that features all sufficient conditions required for the purpose of effective professional development of pedagogical staff [14].

Within our research we consider computer-oriented learning environment (COLE) as a computer-oriented learning environment of institute of postgraduate pedagogical education.

The research goal is to determine the principal factors, criteria, and characteristics, as well as to explain the efficiency evaluation system of COLE.

2 Research Methods

In order to achieve the abovementioned goal, the following research methods have been used: theoretical (analysis of philosophical, psychometric, methodological, specialized literature on the issue under investigation as well as regulatory documentation on the teaching process organization at higher educational establishments and the development of education in Ukraine; identification, analysis, and systematization of the criteria and characteristics of efficiency assessment of COLE); empirical (interviews with students of an institute of postgraduate pedagogical education; direct, indirect, and participant observation of learning process of teaching staff during in-service teacher training in COLE).

Expert evaluation method was used to estimate the numeric value of coefficient of importance of every criterion characteristic of COLE efficiency. A questionnaire [13] was developed on Google Forms basis. This questionnaire was used in an expert evaluation experiment. The subjects have been selected among the most qualified instructors in ICT-related disciplines, coming from the following educational establishments: of Vinnytsia Regional Institute of Postgraduate Pedagogical Education, Donetsk Regional Institute of Postgraduate Pedagogical Education, Zaporizhia Regional Institute of Postgraduate Pedagogical Education, Municipal Higher Educational Establishment Zhytomyr Regional Institute of Postgraduate Pedagogical Education, Municipal Higher Educational Establishment Vasyl Sukhomlynskyi Kirovohrad Regional Institute of Postgraduate Pedagogical Education, Luhansk Regional Institute of Postgraduate Pedagogical Education, Teacher Resource Complex Institute of Postgraduate Education of the National Technical University of Ukraine Kyiv Polytechnic Institute, K.D. Ushynskyi Chernihiv Regional Institute of Postgraduate Pedagogical Education.

It should be mentioned that group evaluation can only be considered sufficiently valid when the opinion consistency among all the experts is very good. Therefore, statistical processing of data received from the experts also contains the evaluation of opinion consistency among the experts, which was measured using the concordance coefficient ${\text{W}}$ [‎2].

Based on the determined coefficients of importance of characteristics ($V_{{p_{yji} }}$), the coefficients of importance of the corresponding criteria ($V_{{k_{yj} }}$) and factors ($V_{{{\text{f}}_{y} }}$) of efficiency assessment of COLE were calculated (see Table 1), where indices $y,\,j,\,i$ indicate the relation of various parameters to a corresponding factor, criterion or characteristic.

Table 1. Criteria factor model of efficiency assessment of COLE.

3 Basic Notions and Definitions

First of all, to conduct efficiency assessment of COLE, the notion of ‘efficiency of COLE’ should be defined.

Dictionaries say that the notion ‘efficiency’ means ‘something that produces an effect (result)’ [23], ‘something that brings required results, outcomes, has the biggest effect’ [8], ‘successful in reaching the set goals’ [18].

A.M. Dolgorukov says that ‘efficiency’ is a judgemental category which is a subjective measure that can vary with the change of our judgments [9].

Amitai Etzioni believes that ‘efficiency of an organization is defined by the level of realization of its goals’ [1].

R.L. Daft stresses that ‘efficiency’ is a broad notion, the use of which requires the consideration of defined variables that characterize individual departments and the entire establishment. Work efficiency correlates with the internal environment of an establishment, which means that its past achievements should be taken into account while setting new goals and defining future strategic course [7].

V.H. Kremen` and V.Yu. Bykov provide a rationalization for measuring ‘learning environment efficiency’ by the degree of correspondence between the quality and quantity characteristics of the created or planned learning environment and the specific target functions according to which it was designed and continues to evolve [16].

Therefore, the term ‘efficiency of computer-oriented learning environment of an institute of postgraduate pedagogical education’ means a judgmental category of COLE, that is defined by the level of realization of its tasks leading towards the main goal of the system of postgraduate pedagogical education and is conditioned by the key characteristics of individual structural components and COLE in general.

According to dictionaries, term ‘factor’ (from German faktor, Latin factor - the one who acts, creates) means cause, driving force of any change, event [10] or process that defines its nature or certain features [6].

Since the key task of COLE is creation and provision of the required and sufficient conditions to the students of in-service teacher training courses facilitating their personal and professional growth, the efficiency assessment of COLE should be carried out through evaluation of determinant factors of COLE, namely: (1) the efficiency of LP of in-service teacher training in COLE; (2) COLE infrastructure sufficiency; (3) ICT competence of COLE academic staff (research and teaching); (4) ICT competence of students.

The measurement of every COLE factor in its turn requires the institute of corresponding criteria for the assessment of COLE efficiency.

Term ‘criterion’ (from Greek kriterion, means of judgment) is defined in dictionaries as ‘a feature, reason for evaluation taken as a basis of classification’ [3]; ‘a feature, sign, which serves as a basis for judgment, means of verification, measure of assessment; in the theory of cognition it is a feature that allows distinguishing between true and false and makes the judgment possible’ [24].

In The Encyclopedia of Education, ‘quality criteria of learning’ is defined as ‘features determining the degree of correspondence of learning to the set goals, standards and norms’ [15].

The National Education Glossary of Higher Education defines the notion of assessment criteria regarding the compliance with educational and didactic requirements as ‘descriptions of what should be accomplished for achieving learning results and at what level’ [17].

Thus, the notion of ‘criteria of efficiency assessment of COLE’ signifies the descriptions of features of COLE, on the basis of which the efficiency assessment of the demonstrated quality of in-service teacher training provision in COLE is carried out.

A corresponding system of criterion characteristics is the manifestation of every efficiency assessment criterion in COLE at a certain development stage of COLE.

4 Criteria Factor Model

The need for objective efficiency assessment of COLE requires the use of qualimetry approach, which envisages comprehensive development and description of methodology of quantitative and qualitative assessment of COLE efficiency.

In order to perform this measurement, the criteria factor model of assessment of COLE should be designed (Table 1), which, in the first place, requires the selection of factors characterizing COLE, their criteria, and the determination of coefficients of importance of factors and criteria.

Decomposition of criteria determined within the criteria factor model of COLE efficiency assessment (Table 1) to the corresponding characteristics selected according to the didactic requirements of COLE [12] will allow to display qualitative and quantitative changes of COLE, to check the compliance and establish the connection between the levels of criteria, factors and characteristics of assessment and efficiency of COLE in general.

Decomposition of principal efficiency assessment criteria of COLE to the corresponding characteristics will be done factor by factor.

4.1 Learning Process

Let us pay a closer attention to the criteria characteristics based on the efficiency factor of learning process of in-service teacher training.

The basis of learning process of in-service teacher training in COLE consists of ‘a system of organizational and didactic measures aimed at the realization of education at a certain qualification level according to the state standards of education’ [22] and modern prospective tendencies related, first of all, to the intensification of LP while using pedagogically considered information and communications technologies.

Therefore, the efficiency of learning process in general is estimated by the integrated characteristic of the system of organizational and didactic measures of LP in COLE aimed at the proper improvement of skills of the teaching staff.

First of all, learning process of in-service teacher training courses of COLE should take place according to the set goals and tasks of COLE. Thus, it is crucial for the organization of LP to estimate the number of students able to take an in-service teacher training course in COLE over an academic year. The key role during the process of academic group formation is played by the correlation between the number of potential students, based on the preliminary recommendation of town and district education boards, to the optimal capacity of COLE in terms of training a certain number of students on the in-service teacher training courses. Personal and professional features of the students, such as special field, qualification category, etc. should be considered. In accordance with this, the academic load is formed, and topics for classes, techniques and methods of successful LP and courses time frame are approved.

The efficiency of learning process in COLE also depends on its compliance with didactic requirements of COLE, namely [12]:

• creation of comfortable, healthy and life saving conditions facilitating the influence on the students’ professional improvement;

• consideration of professional and personal features of students;

• general tasks of LP, development of professional competencies of students in COLE should comply with the goals and tasks of the high-school teaching process;

• creation of conditions for efficient realization, support and control of the unassisted work of students of in-service teacher training;

• creation of conditions for transition from the existing partial use of ICT to the efficient systematic application and development of COLE while using various forms of organization of LP;

• optimization of innovative approaches during LP of in-service teacher training;

• implementation and rational use of the credit-unit system to carry out interim and final monitoring of students’ progress on in-service teacher training courses on the basis of state standards and with the consideration of special regional features of teaching work;

• rational use of distance education elements while organizing LP of in-service teacher training for full-time, part-time, or external studies, or a combination of such;

• integration of individual, micro-group, group, collective and mass organization forms of LP, the basis of which is provision of in-service teacher training courses in group form;

• transformation of the teacher (teaching methods specialist) as a source of knowledge into the knowledge manager, organizer, coordinator, consultant;

• creation and constant update of subject matter of COLE as well as providing students of the in-service teacher training courses with an access to the topical educational information and resources of professional interest.

These requirements determine basic characteristics of the goal corresponding criterion (1.1) of efficiency assessment of learning process of in-service teacher training in COLE (Table 2).

Table 2.  Characteristics of goal corresponding criterion.

LCP efficiency in COLE to a large extent is defined by the motivational level of in-service teacher training participants who want to master practical ICT skills.

4.2 Motivation and Incentives

Let us single out basic characteristics of the incentive motivation criterion (1.2) of efficiency assessment of learning process of in-service teacher training in COLE (Table 3).

Table 3. Characteristics of the incentive motivation criterion.

Organizational and methodological work in learning process of in-service teacher training in COLE, as one of the basic factors of COLE efficiency, is determined, first of all, by adhering to the set course period. It characterizes the quality of organizational and methodological work planning. Particularly, it concerns the level of COLE standardized documents that determine organizational measures of LP of in-service teacher training. Among them, curriculum is the most important.

4.3 Characteristics of Organization and Tool Support

The correspondence of technologies, content, forms, and methods involved in LP of in-service teacher training to the planned ones determines the quality level of academic work, its justification, planning and implementation of learning process in COLE.

With the help of open access to electronic learning resources, such as libraries, professional editions and depository, etc. in COLE, the participants of LP are directed through information environment. They can choose the most suitable educational information, ICT means, etc. for further use in learning, cognitive and professional activities.

A significant amount of time in LP of in-service teacher training is dedicated to the student’s individual work including project preparation, essays writing, and other tasks that require a clear and easy system of proper consulting and reviewing, especially during webinars, phone calls, textual and video communication, etc.

COLE academic staff should handle many tasks that require clear planning and periodic reports. They are generated by significant teaching, methodological, academic and organizational work that should be done by COLE academic staff. The staff also should quickly and thoroughly master the latest ICT means, assess their suitability for learning process of in-service teacher training and at a general education institute. For this reason, academic staff and other workers at higher educational institutes should prepare individual plans and make periodic reports on work they have done during school year. However, such type of planning and reports often repeat the information that can be found in different documents, particularly in the individual plan of the academic worker and their reports on the planned work, or combined work planned and done by the faculty, department or postgraduate pedagogical educational institute. This requires automation of reports on the accomplishment of educational, methodological, academic, and organizational work done by the academic staff of COLE. It is reasonable to organize this automation with the help of modern cloud resources offered by Google and Microsoft.

In such a way, the staff can cooperatively prepare documents, draw up and approve plans of educational activities like learning sessions, consulting hours, conferences, webinars, workshops, round-table discussions, etc., save video/audio reports on them, use file databases in further learning and cognitive activity.

So, let us single out the basic characteristics of the organizational and practical criterion (1.3) of efficiency assessment of learning process of in-service teacher training in COLE (Table 4).

Table 4.  Characteristics of the organizational and practical criterion.

4.4 Characteristics of Students Performance Control and Assessment Criterion

To improve the quality of learning process of in-service teacher training in COLE, it is necessary to automate entry-test-, ongoing-, final- and self-assessment of students with the help of dynamic and fully integrated ICT means for supporting LP and performing students’ development assessment. Such measures will significantly reduce the time for assessment, and save it for learning and cognitive activity.

Teaching program and assessment management systems provide LP participants with the access to the students’ performance ratings according to the credit-unit system. They also allow automatic tracking of students’ attendance. Not only does it help gather group performance data, it also helps analyze them online. This will add more information while making decisions that can influence the performance of any student, group, department, educational institute, as well as the whole system of postgraduate pedagogical educational institute.

This way, basic characteristics of control and regulating criterion (1.4) of efficiency assessment of learning process of in-service teacher training in COLE are determined (Table 5).

Table 5.  Characteristics of control and regulating criterion.

4.5 Characteristics of the Health Preservation Criterion

Health is one of the vital resources of personal and professional development for every professional. That is why special role in efficiency assessment of in-service teacher training LP in COLE is allotted to health-preserving criterion (1.5). Its principle characteristics are provided in Table 6.

Table 6. Characteristics of health-preserving criterion.

4.6 Characteristics to Assess the Results of Learning

Learning process of in-service teacher training should focus on its participants. When efficiency assessment of learning process in COLE is conducted, it is reasonable to take into consideration the results of learning and cognitive activity of in-service teacher training and students’ satisfaction of LP in COLE.

This way, the characteristics of result and reflexive criterion (1.6) of efficiency assessment of learning process of in-service teacher training in COLE are determined (Table 7).

Table 7. Characteristics of result and reflexive criterion.

5 Infrastructure Efficacy Facet

Let us proceed to the criteria characteristics according to the COLE infrastructure sufficiency factor.

Infrastructure sufficiency of COLE is characterized by the available space provided for LP during in-service teacher training in COLE, as well essential equipment needed for teaching rooms, timely installation, scheduled and proper routine maintenance of ICT facilities, which is basis for determining the main criteria and characteristics of infrastructure sufficiency of COLE.

5.1 Characteristics of the Financial Criterion

Rapid development of ICT sphere and high practical value of new ICT technologies in learning process trigger the need of COLE for ongoing maintenance, renovations, repair and setting up of new ICT means, other equipment and devices required for teaching and work rooms in COLE. This, in turn, requires financial support. However, demands of COLE could not be satisfied because of low funding which is mostly provided by regional budgets. On the other hand, efficient functioning of COLE could partially cover these necessary expenses, particularly through enrolling some teachers for passing postgraduate courses on a fee basis, or through commercial use of ICT facilities, etc.

This way, basic characteristics of financial (expense and value-added) criterion (2.1) for assessing COLE infrastructure sufficiency are determined (Table 8).

Table 8.  Characteristics of financial (expense and value-added) criterion.

5.2 Characteristics of the Fail-Safe and Backup Criterion

For the proper functioning of COLE, it is highly important to guarantee high-level protection. It is particularly vital to reach the highest level of safety for the integrity of in-service teacher training LP.

This way, the characteristics of fail-safe and backup criterion (2.2) for assessing COLE infrastructure sufficiency are determined (Table 9).

Table 9.  Characteristics of fail-safe and backup criterion.

5.3 Characteristics of the Organization and Management Criterion

Improved efficiency of COLE requires carefully organized infrastructure management system of COLE. Such system will include all subsystems, communication lines, and processes facilitating efficient COLE.

It has determined the characteristics of organization and management criterion (2.3) for assessing infrastructure sufficiency of COLE (Table 10).

Table 10.  Characteristics of organization and management criterion.

5.4 Characteristics of the Information Dissemination Criterion

The main feature of postgraduate pedagogical education system is an immediate reaction to the modern demands of new information society. This system should take them into consideration while solving current and possible tasks of teaching and organizational processes. That is why the infrastructure of an institute of postgraduate pedagogical education should provide participants of in-service teacher training LP with free access to the wide range of high-quality modern teaching, learning, and educational aids. This way, the characteristics of information dissemination criterion (2.4) are determined (Table 11).

Table 11.  Characteristics of information dissemination criterion.

5.5 Characteristics of the Assessment and Reflexive Criterion

COLE infrastructure should create favorable conditions and opportunities for efficient LP. While assessing COLE infrastructure sufficiency, it is reasonable to take into consideration the assessment of COLE infrastructure sufficiency made by students and academic staff on COLE.

This way, the characteristics of assessment and reflexive criterion (2.5) for assessing COLE infrastructure sufficiency are determined (Table 12).

Table 12.  Characteristics of assessment and reflexive criterion.

6 Academic Staff ICT Competence Facet

Let us determine the criteria characteristics based on the factor of ICT competence of COLE academic staff.

6.1 Characteristics of the Quantitative Criterion

Organization and provision of learning process of in-service teacher training is performed by the academic staff of COLE. It means that the efficiency level of COLE depends on the degree of ICT competence of COLE academic staff.

This way, the characteristics of quantitative criterion (3.1) for assessing ICT competence of COLE academic staff are determined (Table 13).

Table 13. Characteristics of quantitative criterion.

6.2 Characteristics of the Appropriateness Criterion

Modern postgraduate pedagogical educational institute should not act as a unitary unit, but as a center of different educational degrees [11]. For this reason, the degree of its academic staff involvement in the teaching process at general educational establishments and higher educational establishments (of a pedagogical kind) determines the degree of improvement of uninterrupted pedagogical education. This way, the characteristics of appropriateness criterion (3.2) of ICT competence of COLE academic staff are determined (Table 14).

Table 14.  Characteristics of appropriateness criterion.

6.3 Characteristics of the Educational and Qualification Criterion

COLE academic staff’s ICT competence is, first of all, defined by the degree of their academic achievements in using information and communications technologies. Obtaining of further education by academic staff of COLE determines its orientation to constant knowledge enrichment, harmonious individual and professional development.

For this reason, let us single out the basic characteristics of educational and qualification criterion (3.4) for assessing ICT qualification of COLE academic staff (Table 15).

Table 15.  Characteristics of educational and qualification criterion.

6.4 Characteristics of the Research and Practice Criterion

Disclosure on the basis of academic and/or organizational and methodological experience, and combination of results of theoretical and practical research of COLE academic staff concerning the use of information and communications technologies in learning process during in-service teacher training, creates regularities of further development of COLE. This way, the characteristics of the research and practice criterion (3.4) for assessing ICT qualification of COLE academic staff are determined (Table 16).

Table 16.  Characteristics of research and practice criterion.

6.5 Characteristics of the Assessment and Reflexive Criterion

It should be mentioned that the level of organizational and teaching work of COLE academic staff is determined by the level of measures relating to teaching work administration, which is done at two levels: at the level of the department as the principal unit of an institute of postgraduate pedagogical education that organizes teaching work around academic subjects; and at the level of postgraduate pedagogical educational institute, where problems are solved and teaching activities of general institutional and inter institutional scale between departments are performed.

The extensive use of the capabilities of information and communications technologies in LP of in-service teacher training requires high level of ICT competence from COLE academic staff.

To determine the level of ICT competence of COLE academic staff, self-evaluation and students’ evaluation of ICT competence of COLE academic staff should be taken into consideration. This way, the basic characteristics of assessment and reflexive criterion (3.5) are determined (Table 17).

Table 17.  Characteristics of assessment and reflexive criterion.

To do a detailed evaluation of ICT competence level of COLE academic staff according to every indicator, it is needed to make a distribution of academic and assisting staff based on their involvement in the postgraduate pedagogical educational institute work: based on the primary place of employment, on the second job, on an hourly basis, which will intensify LP and increase the work level of an institute of postgraduate pedagogical education regular staff, and also become fundamental in staff correction of an institute of postgraduate pedagogical education.

7 Students ICT Competence Facet

Let us determine the criteria characteristics based on the factor of ICT competence of COLE students.

All COLE academic staff, to a greater or a lesser extent, is responsible for the quality of student advanced training: all academic, teaching methods and administrative workers of an institute of postgraduate pedagogical education involved in LP (headed by the department chairs, dean’s office of an institute of postgraduate pedagogical education). That is why defining the level of ICT competence of students passing postgraduate courses has a great importance in evaluating the efficiency of COLE, as well as in decision making by COLE academic staff (that is responsible for students’ training level) regarding the optimality of using information and communications technologies during learning process of in-service teacher training.

7.1 Characteristics of the Motivational and Axiological Criterion

In-service teacher training in an institute of postgraduate pedagogical education is done by individual education plan (drafted on the basis of curriculum). The responsibility for its implementation lies with the students themselves. For this reason, COLE students’ academic success is determined by the level of their motivation and their targeting at the appropriateness of ICT use in learning, cognitive and professional activity.

This way, basic characteristics of motivational and axiological (value-conscious) criterion (4.1) for assessing ICT competence of in-service teacher training students in COLE are determined (Table 18).

Table 18.  Characteristics of motivational and axiological (value-conscious) criterion.

7.2 Characteristics of the Cognitive Criterion

The peculiarity of in-service teacher training in COLE is contribution to the students’ mastering of new knowledge about the basics of functioning and peculiarities of ICT use in education.

For this reason, let us single out basic characteristics of cognitive (knowledge) criterion (4.2) for assessing ICT competence of in-service teacher training students in COLE are determined (Table 19).

Table 19.  Characteristics of cognitive (knowledge) criterion.

7.3 Characteristics of the Praxeological Criterion

The attainment level of in-service teacher training students in COLE also depends on the level of their skills in using ICT at the general educational establishment. This way, basic characteristics of praxeological (pragmatist) criterion (4.3) for assessing ICT competence of in-service teacher training students in COLE are determined (Table 20).

Table 20.  Characteristics of praxeological (pragmatist) criterion.

7.4 Characteristics of the Academic Criterion

When evaluating ICT competence of in-service teacher training students in COLE, it is important to consider the students’ use of information and communications technologies in research and educational activities. For this reason, let us single out the basic characteristics of the academic criterion (4.4) (Table 21).

Table 21.  Characteristics of the academic criterion.

7.5 Characteristics of the Assessment and Reflexive Criterion

The constantly increasing level of ICT competence among in-service teacher training students stimulates the renovation of technologies of professional development of COLE teaching staff.

That is why the characteristics of assessment and reflexive criterion (4.5) of ICT competence evaluation of in-service teacher training students in COLE (Table 22) are among the key parameters of efficiency assessment of COLE.

Table 22.  Characteristics of the assessment and reflexive criterion.

8 Assessment Based on the Proposed Model

On close examination of COLE efficiency on the basis of the built criteria factor model, the expert group needs to determine the levels of each COLE efficiency criterion’s manifestation. It is relevant to do through measurement of appropriate criterion characteristics based on the following system of evaluation: 0 points – the characteristic is not exhibited; 1 point – the characteristic is more not exhibited than exhibited; 2 points – the characteristic is more exhibited than not exhibited; 3 points – the characteristic is exhibited completely. If the coefficient of criterion manifestation ($K_{{k_{yj} }}$), which is defined as a sum of products of assessment of each criterion’s manifestation by their corresponding importance: $K_{{k_{yj} }} = \sum\limits_{i=1}^{n} {\left( {O_{{p_{yji} }} \cdot V_{{p_{yji} }} } \right)}$, where ${\text{n}}$ is the number of $k_{yj}$ criterion characteristics not less than 1.5, then such criterion is considered positive and the level of its manifestation is determined by the numerical value of the $K_{{k_{yj} }}$ coefficient, i.e.: 1,50$\le K_{{k_{yj} }} \le$1,65 – critical level, 1,66$\le K_{{k_{yj} }} \le$2,25 – satisfactory level, 2,26$\le K_{{k_{yj} }} \le$3,00 – high level of criterion manifestation.

If the share of positive criteria of the measurable factor is not less than 50 %, such factor is considered positive. The level of its manifestation is determined based on the numerical value of the factor’s manifestation coefficient ($K_{{f_{y} }}$), which is calculated according to the formula: $K_{{f_{y} }} = \sum\limits_{j = 1}^{m} {\left( {K_{{k_{yj} }} \cdot V_{{k_{yj} }} } \right)}$. If 1,50$\le K_{yj} \le$1,65, then the factor of manifestation is of critical level, with 1,66$\le K_{{f_{y} }} \le$2,25 it is satisfactory, while 2,26$\le K_{{f_{y} }} \le$3,00 means it is of high level.

If in the process of measuring of the efficiency of COLE at least one of its factors is negative, such COLE is considered ineffective. On the contrary, if all the factors of COLE efficiency are positive, such COLE is considered efficient. Its efficiency level is determined through the numerical value of the efficiency coefficient ($K_{e}$): $K_{e} = \sum\limits_{y = 1}^{4} {\left( {K_{{f_{y} }} \cdot V_{{f_{y} }} } \right)}$, – i.e.: 1,5$\le K_{e} \le$1,65 as critical, 1,66$\le K_{e} \le$2,25 as satisfactory or 2,26$\le K_{e} \le$3,00 as high.

Criteria factor model of efficiency assessment of COLE, built this way, allows determine the level of efficiency of COLE in Ukraine.

However, COLE is a complicated educational system. It must correspond both to the state standards and the current needs of the rapidly changing world. Therefore, when conducting criteria factor evaluation of COLE efficiency, we need to correlate it to the average and the highest level of efficiency of computer-oriented learning environment of an institutions of postgraduate pedagogical educational in Ukraine.

9 Conclusions and Future Work

To conclude, the efficiency of COLE is a judgmental category of COLE defined by the level of realization of its tasks leading towards the main goal of the system of postgraduate pedagogical education and conditioned by the key characteristics of individual structural components, and COLE in general.

The need for objective efficiency assessment of COLE determined the development of the criteria factor model and description of the system of COLE efficiency assessment.

The use of the built criteria factor model of efficiency assessment of COLE (Table 1) provides criterion measurement of the efficiency of COLE based on four factors: (1) the efficiency of LP of in-service teacher training in COLE; (2) COLE infrastructure sufficiency; (3) ICT competence of COLE academic staff; (4) students’ ICT competence. To conduct such an assessment 80 criteria were suggested, each of them containing 2–7 characteristics (Tables 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, and 22).

There is a need in further research into the efficiency assessment of COLE in Ukraine. It will allow to identify advantages and disadvantages of computer-oriented learning environment of an institutions of postgraduate pedagogical educational in Ukraine and suggest the desired trends.