Keywords

1 Introduction

Higher Education contributes to the social, economic and political growth of the countries, because it trains future professionals who will generate different solution alternatives aimed at solving social problems. Universities in Colombia materialize this contribution through the definition of policies aligned to the institutional teleological component, and the design and operationalization of action plans that contribute to the improvement of the quality of the training process and of all administrative processes to support it. With the intention of promoting the acquisition and development of both generic and professional skills and competences, [45] proposes a proposal in this regard.

The Colombian university system has implemented different strategies, leading to improve the service offered, as evidenced in [42, 45]. Understanding that their quality, accessibility and efficiency depend both on the economic development of the geographical area and on human capacities [5]. Despite the commitment to quality shown by the Colombian Ministry of National Education (primarily in the last two decades), in terms of definition and implementation of such strategies, many HEIs still do not have effective monitoring and accountability systems (that allow them to evaluate the performance of their strategic areas), which does not contribute to the improvement of the processes and is contrary to the quality criteria required by higher education for the training of qualified professionals.

The use of an appropriate management tool provides the opportunity to increase academic, scientific and cultural quality by facilitating the process of competing with leadership in the increasingly demanding university education market [1]. Given this, it is relevant to have clarity about the internal purposes as an institution and to have alignment with the external requirements or parameters through which the quality of education is determined. In this paper the process of building a system for tracking and monitoring the strategic areas belonging to a university is showed. The paper is made up of five sections: the first section shows an introduction, the second section details the evaluation of the management and quality of processes in HEI, the third section shows the design of the management model and quality of the methodology defined by the management system of the BSC, the fourth section describes the information system that supports the model and each of the modules that constitute it, finally the conclusions and references are presented.

2 Evaluation of the Management and Quality of Processes in HEI

In the High Education Institutions, the evaluation of the management and quality of processes will be strategically addressed from: the analysis of the Indicators system (based on a global perspective), the mechanisms of quality assurance in education (based on the normative reference defined in the Republic of Colombia, by the Ministry of National Education) and the Balance Scorecard (as an integrating tool that facilitates the decision making).

2.1 Indicators System

The management of processes in any organization is a complex task that must be developed under a review approach at both micro and macro level to be able to cover all the needs of the areas that compose the organization. Once management maintains a continuous and permanent task, should be measured the quality of its processes and subsequently take actions that allow to optimize its resources and increase the impact of its activities. The HEI as organizations, are no stranger to this, because the quality of higher education must cover all its functions and activities with reference to teaching, programs, research, staff, students, infrastructure and services to the community and the university world, as raised [6]. This implies, that management in education integrates policies in a practical way through the purposes of the organization, by means of planning its procedures and the permanent evaluation of these [2]. It also implies that those responsible for carrying out the management, recognize the need to treat the information at the inputs, during its processing and at the outputs, so as to facilitate decision-making in coherence with the purposes of the university, according to [3].

Educational planners must pay attention to quality [3], so that strategic planning and performance evaluation play an essential role in consolidating a better future for society [4]. Recognition of key performance indicators is one of the main steps in performance evaluation [7]. Indicators have become an essential tool for describing and understanding the quality of a system. They are instruments of observation and monitoring of a system, designed from the relation of variables of the system. The measurement of these variables and their subsequent comparison with the established goals, allows to determine the achievement of the system and its trend of evolution [8].

From the cybernetics and control concepts, described in [8], the following steps are defined to establish an indicator:

  • Have objectives and strategies

  • Identify critical success factors

  • Establish indicators for each critical success factor

  • Determine for each indicator, the status, the threshold and the management range

  • Design the measurement and source of information.

In [6] defines the “System of Indicators”, within the context of education, as the coherent set of indicators, combined or not, according to a system of variables and categories that represent the management or operation of a unit of analysis for a given function, for example: teaching, research, extension or institutional service.

The evaluation of higher education systems and the measurement of the objectives achieved is a complex task. For this reason, many measurement methods have been proposed with opinions differ on which are the most appropriate indicator systems, some of those proposals are: [10,11,12,13,14]. These tools have been designed to perform or support certain functions and the debate focuses more on its use than on the way they are designed and implemented. In [15] and [16] indicators structures have been proposed, in order to have a more organized construction and obtain a very close and reliable representation of the interests and projection of the organization. These structures have grouped indicators considering results, internal organizational processes, integrative criteria, organizational culture, and capacity for change, linkage between resources and results, technical aspects of the organization, and relationship of the organization to human factors.

In Latin America has been observed in a significant use of indicators, according to the quality of: curriculum components, process related to educational management (human resources, material resources and didactic factors), immediate results related to the training acquired by students, services and integration [3]. Such indicators have been categorized in academic, research and support, and based on the characteristics of the Analytical Hierarchy Process (AHP), trend analysis and comparative data, as manifested in [17]. In Colombia, a recent study presents a list of indicators related to processes and activities carried out in planning, teaching, research, social responsibility, welfare, internationalization, management and resources [18]. On the other hand, there are evaluation tools that have been used by the education sector to maintain sustainable monitoring and benchmarking [19].

In the European continent, the application of indicators has developed much more. Since the end of the twentieth century, different universities have implemented information management systems based in indicators, which support to decision making in the development of academic processes that were mainly oriented toward teaching, research and management. The above, has facilitated the integration of indicators with a set of quality improvement tools, that were developed in subsequent years, as evidenced in [20,21,22]. It should be noted that there is no marked trend in the use of any quality management tool. However, the philosophy of Total Quality Management (TQM) has been used as a conduit to achieve the objectives of Quality Management for organizations [24].

Asia has not been alien to these structures, considering that BSC-based indicators have been constructed for university education centers through information systems and academic monitoring, as referenced in [25,26,27,28].

In different world scenarios, proposals are being developed to strengthen the measurement of quality in Universities. In [46] the interaction between the organizational structure/managerial and organizational value/ psychological elements that impact on the quality of education was evaluated. Based on this, a route analysis was carried out on the data collected from the academics with teaching coordination functions. The study recommended the creation of policies and institutional strategies aimed at improving educational quality through the consolidation of collaborative teaching/learning communities with an explicit concern for morality, participation and development. In [47] a committee for quality assurance (QAC) is proposed within Italian universities in order to identify if, as happened in other New Public Management (NPM) reforms, to detect the key variables that promote a satisfactory QAC functioning; based on both the analysis of the composition and the role of QAC in all the Italian public universities and significant case studies.

2.2 Mechanisms of Quality Assurance in Education

When an organization refers to the term of quality, it is associated to the satisfaction of the expectations according to defined criteria that are objectively evaluated, allowing to have greater proximity to degrees of excellence. The concept of quality in education has arisen because governments, educational institutions and society have identified the importance of permanently improving academic training processes that lead to a better economic development of society.

The concept of quality applied to the public service of higher education refers to the synthesis of characteristics that allow to recognize a specific academic program or institution of a certain type and make a judgment about the closeness between how the institution or the academic program provides this service and how it should be provided [29]. This situation has given rise to a growing social requirement to improve and ensure the quality of universities and their undergraduate and graduate programs. In response to the above, both the State and the institutions themselves have generated quality assurance mechanisms, in order to give greater guarantees to users and the general public regarding compliance with minimum standards of quality and the levels of performance of graduates [30].

Colombia began its experience of quality improvement of higher education through the accreditation of high quality that appears in Law 30 of 1992 [31]. Taking into account the requirements of the society, the mechanism was created to obtain a qualified register of obligatory character, coordinated by the National Commission for the Quality Assurance of Higher Education (Comisión Nacional de Aseguramiento de la Calidad de la Educación Superior - CONACES); and guidelines for high-quality accreditation of voluntary nature governed by the National Accreditation Council (Consejo Nacional de Acreditación - CNA).

In order to offer and to develop an academic program of higher education, in the address of an HEI, or in another place, it is necessary to have previously the qualified registry of the mentioned program [32]. While the central purpose of accreditation is to promote continuous improvement and to determine whether an academic institution has quality in general or respect of one or more of its careers or educational programs, if it is able to demonstrate that it is progressing continuously and systematically with the use of adequate strategies, procedures and resources to achieve its mission and objectives, reasonably fulfilling the established criteria and quality standards [33].

In most of the countries at the global level, national accreditation systems are created, which seek to ensure that HEIs that are part of these systems meet the highest quality requirements to offer programs that can meet the labor demand. In Colombia, the accreditation process does not arise within the framework of state inspection and oversight, but rather in the promotion, recognition and continuous improvement of quality [34]. Considering the above, it can be affirmed that both the obtaining of the qualified registry and the guidelines for the high accreditation, seek the recognition and the quality of the academic programs. The first is the way to achieve the basic operating conditions for the offering of the programs and the second continually seeks to improve processes to achieve academic excellence.

2.3 Balance Scorecard

In 1992 Kaplan and Norton of Harvard University, revolutionized business management with a proposal known as the Balance Scorecard (BSC) to align the company towards achieving organizational strategies through tangible goals and indicators. Kaplan states that “Managers, like pilots, need an instrument that measures their environment and performance to lead the journey towards future excellence”.

The BSC is a management tool that assists decision making, by providing periodic information on the level of compliance with the objectives, previously established through indicators, the latter include both financial and non-financial aspects. The BSC favors transparency in management and the establishment of a balance between immediate actions and strategic lines, by integrating four perspectives or key areas and relating them to the mission, vision and objectives. The four perspectives are financial, training, internal processes and the relationship with customers/users.

In the BSC, it is preferable that the indicators be of a numerical nature, as this will allow the establishment of tolerance levels. According to their nature, there are indicators that measure: efficiency, economy, effectiveness, excellence and environment [35]. Management through performance measurement, has historically been an important aid in enabling managers to diagnose a situation and learn more about it. In the 1960s and 1970s, Target Management became a widely used management tool to align management actions with organizational objectives. In addition, BSC has a huge impact on knowledge creation and provides greater interactivity, since it helps to communicate strategy involving different levels of the organization [36]. On the other hand, the Balanced Scorecard provides models and processes for measuring and supervising the performance of human resources and their impact on the strategic success of the company [37].

For example, Universidad Centroccidental Lisandro Alvarado (UCLA), in the implementation of its BSC has established as perspectives the state, society, internal processes, and organizational development and learning. Its implementation was supported on the systems of control of management like accounting and budget, taking into account that the institution belongs to the public sector [38].

The BSC proposes to obtain relevant information about the main factors that can lead to the achievement of the objectives of the universities. It is also very useful for communicating the strategy to the entire university community and for the goals of each employee to be consistent with those of the university itself [39].

The implementation of a BSC through information systems, provide organization, dynamism and decrease the margin of error in making decisions, by enhancing the analytical, organizational, operational and financial capacities of each company. On the other hand, it provides the environment, structure and language to communicate mission and strategy, using measurements to inform employees about the causes of current and future success [40]. Organizations are becoming increasingly sensitive to the need for management information systems, largely due to the changing environment and globalization. In addition, universities have a complex organizational structure, characterized by a high dispersion of authority for decision making in various bodies. Therefore, the BSC reaches its maximum expression when it is designed and implemented through the use of new information technologies [41].

3 Design of the Management Model and Quality

It is important to highlight that the methodology defined by the management system of the BSC was established for the construction of indicators. For the review of the tracing and monitoring systems applied in HEIs, it was considered that they would have the recognition of institutional accreditation that guarantees an academic organization with high quality standards. In this way the different indicators created according to each type of institution were identified, reflecting the characteristic situation of the institution and the approach to which each one contributed. Subsequently the requirements defined for the design of the system were determined, establishing the starting point for the creation of the technological tool. The classification of these requirements was relevant to define in more detail the types of requirements necessary to have accurate and truthful information of the system.

In order to construct an informative diagnosis in relation to the current measurement system of the HEI studied, the strategic map scheme was designed, which consolidated the structure of the institution’s vision in relation to its processes and procedures. Subsequently, the required indicators were elaborated according to the objectives established in the perspectives defined in the strategic map, thus relating indicators-objectives-perspectives. For each indicator the following items were defined: the associated objective, name, display order, description, formula, unit of measure, orientation (if desired to maximize or minimize), tolerance levels, target, frequency of collection, information source, data quality, periodicity of the goal, accreditation factor, collection manager (s), performance manager (s), version, version update date.

With the articulation between: the Institutional Educational Project (IEP), the Program Educational Project (PEP), the interests of the management, the national regulations established for minimum quality conditions for higher education programs, and the policies established by the National System of Accreditation, it was possible to construction a strategic map as shown in the Fig. 1, which grouped and organized in a structured way the main objectives in order to obtain concrete results that generate value in the HEI.

Fig. 1.
figure 1

University strategic map

Full size image

The perspective of Academic Processes is related to the integrality and flexibility of the curriculum, interdisciplinarity, teaching and learning strategies, the student evaluation system, and academic and teaching support resources. The Visibility perspective focuses on the insertion of the program in national and international academic contexts, for which it seeks to increase the visibility of the University at a national and international level, and strengthen the academic and administrative management of the graduate Department. The Self-regulation perspective aims at the permanent evaluation of the programs through the existence of an efficient, effective and sustainable management of resources, articulated with the IEP. The Research perspective ensures that high quality programs, according to their nature, are recognized for the effectiveness of their research training processes, and for their contributions to scientific knowledge and innovation, an interesting implementation of this perspective is presented in [44]. Finally, the Extension perspective is related to the positive influence of academic programs on their environment, promoting the link with the various sectors of society.

To achieve the objectives, 63 indicators were divided among the five perspectives previously mentioned. These indicators are obtained and measured by users in the critical areas of Academic Management, Internship Management, Financial Management, Publications Management, Research and Innovation Management, and Administrative Management.

When consolidating the indicators, it was observed that the perspectives of Academic Processes and Research represent 51% and 35% of the measurement system, respectively. This indicates that the HEIs currently concentrate their efforts on maintaining a quality education by guaranteeing procedures and adequate resources for both students and the teaching staff so that updated tools and corresponding competencies are provided for adequate job placement.

4 Information System

The model of management and quality of the university processes was implemented through a software, which integrates: a transactional system for the recording of the measurements of the indicators and an interface with visual reports for the decisions making. The Information System includes modules that allow: the parameterization of the software, the management and the generation of reports for decision making. The main interface presents a web desktop (see Fig. 2), to access different windows and view essential information of the BSC through widgets. In addition, it has a main menu to access the different functionalities of the software.

Fig. 2.
figure 2

Screenshot of login and web desktop

Full size image

The desktop allows the opening of several windows at the same time and navigate among them through the web desktop. The default view of the web desktop is enabled for the visualization of widgets and direct access to the functions most used by the logged in user.

4.1 Parameterization Module

The information system allows the definition of different parameters, such as the assignment and denial of permits, for the access of user groups to the different modules. Similarly, within the system, user profiles and all their associated information are managed. Areas of university and academic programs can also be defined. On the other hand, for the implementation of the BSC is necessary to make the configuration of the strategic map. First, the perspectives are defined and then the objectives associated with each of them.

The life cycle of an indicator consists of seven moments, from its creation and configuration to its measurement and visualization, these moments are: creation of the indicator, definition of goals, definition of performance managers, definition of collection manager, activation of the indicator, recording of measurements and visualization of the indicator.

4.2 Management Module

At the transactional level, this module is important, since it allows to carry out the measurement records of each one of the indicators by each academic program. It means, the value that an indicator can take at a specific time may be different for each academic program. Depending on the user who has logged into the platform, the indicators that can be obtained will be displayed, presenting them in a panel according to the frequency of the collection. Depending on the formula of the indicator, the user will see how to enter it. Future periods cannot be recorded, but it is possible to insert all periods prior to the current date. After three (3) days of the indicator registration deadline, an alert will be generated by email to the user. In addition, for each measurement period the user can attach a support document (Fig. 3).

Fig. 3.
figure 3

Screenshot of indicators entry

Full size image

4.3 Reports Module for Decisions Making

Within the information system, three key reports can be found in order to trace and make decisions based on the results of the management: the strategic map, the history of the indicators and the current state of the indicators.

The strategic map lists the defined perspectives, the objectives of each perspective and the indicators of each objective. For each indicator its current value or state can be seen, through a semaphore (red: bad, Yellow: normal, Green: fine), each time the indicator is updated, the last measurement period and other indicator information is displayed. This panel serves to identify bad indicators and to know what objective is not being met in the institution. Reporting is done in two layers, in the first layer the measurements can be seen at the institutional level, but also can be seen the detail of each indicator, identifying the measurement of that indicator by program. In this way, it can be known, which program is lowering or increasing an indicator, to make decisions about it.

On the other hand, there is a report on the indicators history, where the different measures that an indicator has had throughout its existence are displayed on a graph. This allows to know if the indicator is growing or decreasing, in order to take corrective measures or to design strategies that allow to achieve the goals. For this reason, in this report the goal to be achieved can be seen, in each one of the graphs of the indicator. Finally, Fig. 4 shows the report of the current status of the indicators, where a semaphore is displayed that indicates the limits of the tolerance ranges of the indicator and its current measurement.

Fig. 4.
figure 4

Screenshot of indicators display (Color figure online)

Full size image

5 Conclusions

All HEI requires a tool to measure and control its processes in order to evaluate the impact of its activities and compete in the market. For this, it requires great support and commitment from the areas involved for the process of constructing indicators that reflect their need and maintain the objectivity of the evaluation.

There is no accurate measure of indicators to evaluate a university, but there are references of quality that guide the creation of these towards compliance and control of processes. In this way national accreditation guidelines and guidelines for obtaining qualified registration are an essential starting point for effective measures in the education sector.

In designing the tracing and monitoring model, it was possible to define that the main purposes in which the system should be focused were to achieve an integral formation in the students of the different academic programs of the University, in line with the challenges of high quality, by strengthening: the teachers’ plant (consolidating the programs of admission, promotion, permanence and graduation of the student population with relevance and quality, and increase the visibility of the University at national and international level), the academic and administrative management of the graduate department (keeping an efficient, effective and sustainable management of resources, articulated with the IEP), the research process (in coherence with the IEP and the needs of the environment) and increase the relevance and social impact of the Institution.

At the same time, the most relevant perspectives that concentrate the functionality of the HEI are the Academic Processes and the Research, concentrating the Universities their efforts in maintaining a quality education by guaranteeing procedures and resources that are appropriate both for students and the teaching staff so that updated tools and corresponding competencies are provided for adequate job placement. Also, it is of great importance to maintain a continuous process of knowledge flow that allows to generate the evolution and the transcendence of the science. Finally, the importance of the use of an information system for the implementation of educational quality measurement systems is highlighted, considering that it is necessary to have three indispensable requirements for decision making: integrity, reliability and availability.