Keywords

1 Introduction

According to UNESCO, education is the process of facilitating learning or the acquisition of knowledge, skills, values, beliefs and habits (UNESCO SDG, 2022). The main goal of education is to ensure that students have all that is necessary to become a successful, self-reliant, productive and virtuous member of society. But what is absolutely necessary? Success in the labor market depends on a number of conditions. Some of these are conditions over which the individual has very little control. For example, the global economic situation, or the global health situation (see the current pandemic) are all factors that determine job placement. These are factors over which not only the individual, but also the education system has only a negligible influence, and are therefore not the focus of this study. We focus our attention on the individual's own inner qualities and most important characteristics.

As UNESCO stated, education should be responsible for the acquisition of necessary abilities, skills and knowledge to become efficient in the labor market. However, the quality of education in a country is not at all the same in different regions, educational disadvantage is present almost everywhere. It usually includes inequalities in educational outcomes, which can seriously hinder the successful employment. Is it possible for people who live in a disadvantaged area to possess all the abilities and skills that is required? This is a hard question to answer, because it is not easy to determine exactly what should a worker have and what has a person living in a disadvantaged region? A theory-based methodology and measurement tool is needed to answer this question.

In this paper we look at this issue from a labor market perspective. We first define what might be needed in the labor market, presenting a theoretical framework, then we discuss the effectiveness of a training programme supported by a digital measurement system in a disadvantaged Hungarian region. The programme lasted for approximately two years and was designed to reintegrate disadvantaged people who, for various reasons, had not received an adequate education or had dropped out of the education system, back into the world of work.

1.1 What is Needed at the Workplace?

Currently there are almost countless occupations, and according to The Occupational Information Network (O*NET), which was developed under the sponsorship of the U.S. Department of Labor/Employment and Training Administration (USDOL/ETA) “every occupation requires a different mix of knowledge, skills, and abilities, and is performed using a variety of activities and tasks”. But do they have anything in common?

According to researchers and HR professionals, cognitive ability is one of the most important factors in predicting job performance (Grobelny 2018). However, there is an ongoing debate in the literature about whether general or specific cognitive abilities predict job performance better. Human resource management researchers generally view cognitive ability as a one-dimensional construct. This is because many believe that general cognitive ability has relevance for selection (Schmidt and Hunter 1998, 2004) and that measuring specific cognitive ability in a narrower domain is not worth the time and effort (e.g. Hunter 1986). These claims are largely supported by meta-analyses based on studies of the validity of general ability tests and incremental validity analyses using hierarchical regression models (e.g., Carretta and Ree 1997; Schmidt and Hunter 1998). The use of a one-dimensional approach in the human resources field is convenient from a practical point of view, as it offers a simple and parsimonious solution that does provide a not too bad estimate (Schneider and Newman 2015).

The mainstream approach emphasizing the predictive role of general cognitive ability is based on three main arguments: 1) general cognitive ability is the single most valid predictor of job performance, 2) the predictive validity of general cognitive ability is independent of the occupational context, and 3) specific cognitive ability is not expected to have incremental validity (Schmidt 2002). However, there are doubts about all three claims. The evidence supporting the higher predictive validity of general cognitive ability comes mainly from meta-analysis-based studies. These types of studies summarize the results of the validity of different tests from hundreds of samples, usually highlighting a single main coefficient and pooling the results of several studies conducted under different conditions in different contexts. As a result, a large amount of contextual information is lost due to generalization.

In addition, there have been several conflicting results that have not shown any predictive power of general cognitive ability on job performance, e.g., in law enforcement occupations (Hirsh et al. 1986), salespeople (Hogan et al. 1992; Verbeke et al. 2008), bankers (Barros et al. 2014), and insurance brokers (Downey et al. 2011). However, some specific cognitive abilities have been shown to be reliable predictors (with higher validity than general cognitive ability) in some contexts, e.g., mechanical understanding and reasoning in manufacturing workers (Muchinsky 1993), performance speed in office workers (Whetzel et al. 2011), and perceptual speed in warehouse workers (Mount et al. 2008). According to Krumm et al. (2014), recent meta-analysis-based studies suggest that specific cognitive abilities can not only complement but also surpass the predictive power of general cognitive ability.

There are a number of factors that may moderate the predictive power of both general and specific cognitive abilities on job performance. On the one hand, if specific cognitive ability scores are calculated according to specified criteria, estimated specifically for the occupational group under study, they are responsible for a significant part of the variance in job performance. (e.g. Schneider and Newman 2015). General cognitive ability, on the other hand, loses its predictive power for job performance when other attributes, such as social skills, are also taken into account (e.g. Schneider and Newman 2015). Third, for performance measures that are more reliable than supervisor ratings, general cognitive ability has been found to be only a weak or non-significant predictor (e.g., La Grange and Roodt, 2007).

According to Schneider and Newmann’s (2015) compatibility principle on the relationship between cognitive abilities and job performance, general cognitive abilities predict general job performance, while specific cognitive abilities predict specific job performance. This principle is based on both the theoretical work of Ajzen and Fishbein (1977, 1980) on attitude-behavior compatibility and empirical research on the predictive validity of specific cognitive abilities (e.g., Reeve 2004; Hunter 1986; Joseph and Newman 2010). Grobelny made a similar assumption based on (Motowidlo 1997), Borman and Schmidt's (1997) theory of individual differences in job performance. According to this theory, variance in job performance is caused by variation in characteristic adaptations. Characteristic adaptations are the result of specific skills and patterns in workers’ behaviors, individual differences in personality and abilities, and interactions with learning experiences from the environment. They represent the implementation of specific behaviors that are necessary to perform a particular job, and the implementation of specific behaviors can be linked to specific narrow domain skills (Sternberg 2001). Within particular occupations, similar tasks and behaviors are generally required to be performed, and thus a defining specific cognitive ability can be assumed to underlie them. In contrast, general cognitive ability plays a role in general problem solving and functioning, which may play a role in predicting performance, but not to the same extent as specific abilities. Since different occupations differ significantly in the actions they require, they also differ in the specific adaptations they require. These suggest that the predictive validity of specific abilities also varies widely across occupations. Grobelny’s (2018) own research backed up his ideas. Specific cognitive abilities had higher predictive power than general cognitive ability, and the predictive power of different abilities depended on the job in question.

Based on these theoretical and empirical studies we conclude that it is more useful to test specific cognitive abilities in a job performance assessment because they are more predictive of job performance. We assume that this also means that specific cognitive abilities should play a more prominent role in school education, because they lead to a potentially more successful employment. Because of this, their study is therefore warranted. However, in order to know what needs to be examined, a theoretically sound taxonomy is necessary. The Cattel-Horn-Carroll model of intelligence is one of the most obvious options, as it is currently considered the most widely accepted model and the one best supported by empirical research (Sternberg 2012). The Cattel-Horn-Carroll model claims that there is a large number of various cognitive abilities, and they can be grouped into 3 strata: stratum I, “narrow” abilities; stratum II, “broad abilities”; and stratum III, consisting of a single “general ability” (or g). Below the g there are eight broad abilities: Comprehension-Knowledge (Gc), Fluid reasoning (Gf), Quantitative knowledge (Gq), Reading & Writing Ability (Grw), Short-Term Memory (Gsm), Long-Term Storage and Retrieval (Glr), Visual Processing (Gv), Auditory Processing (Ga),Processing Speed (Gs). A number of extensions to CHC theory was also proposed, including Domain-specific knowledge (Gkn), Psychomotor ability (Gp)and Psychomotor speed (Gps). A full review about the model can be found in Schneider and McGrew’s (2012) study.

1.2 Education in Disadvantaged Regions

Presenting all the issues with the education in underdeveloped regions is beyond the scope of this paper, however, we would like to note a few important points. Most countries have areas that are underdeveloped compared to other regions of the country. Such regions are characterized by poverty, less developed educational infrastructure and few job opportunities. There are several aspects of educational disadvantages: high teacher turnover, low retention rates, less confidence in the benefits of education, limited cultural facilities in the community, lack of employment opportunities for school completers, and a less relevant curriculum (Lamb et al. 2014). Students in underdeveloped areas perform worse than students in developed areas, student reading literacy and school learning environments are less positive (Sullivan et al. 2018).

Digital education is difficult to implement in these regions, where families often do not have access to the internet or digital tools. Digital illiteracy is in itself a disadvantage in the labor market, where digital technologies are becoming more prevalent, and the lack of access to smart tools can create a negative attitude towards digitalization. Considering the trend that in many places digital assessment procedures are used in the selection of employees, it is also possible that the lack of digital skills and tools may lead to underperformance in tests that are digital.

This is why we need to be very careful when using digital tools to teach them and assess their performance. In every assessment it is important that the procedure is sufficiently standardized to minimize potential differences, and this is particularly important for people living in underdeveloped regions. We need a procedure that is easy for them to use, standardized and that gives them the opportunity to demonstrate their real abilities.

2 Training Programme for Disadvantaged People

2.1 Training Participants

The goal of the training was to reintegrate disadvantaged people into the labor market. The European Code of Conduct for Research Integrity and the Ethical Principles of the Hungarian Psychological Society have been fully taken into account in the development of the programme. 120 people participated in the programme. The average age was 35 years and the sex ratio was approximately equal. Participants volunteered for the programme and could count on the help of an inclusion mentor throughout.

2.2 The Programme Process

The programme started with a pilot study in 2018 to identify the requirements for those jobs in the region that can be filled without higher education. These are the jobs in which participants have the best chance of finding a job. As part of the pilot study, we carried out a skills assessment with a digital system in two large companies in the region, mainly looking for people to work on the production line. The results of this assessment served as reference (N = 120), to which we could compare the results of the training participants.

After the pilot study, we carried out the assessment (which began in 2019) of the participants, using the same digital tools as in the pilot study. The assessment took place in 6 different settlements, in groups of 20 people. The purpose of this was to take stock of the situation and to help identify further training directions. The initial measurement was followed by a 30-h training which aimed to develop those areas that required development. The choice of areas to be developed was subject to certain constraints. The aim was not to develop knowledge, and this training is not the most appropriate for this purpose. It was also not intended to bring about changes in personality traits, as such a result could not be expected from a 30-h training course. The focus was on abilities and skills that could be changed in a meaningful way. During the training, participants were given tasks that tested their logical thinking or manual dexterity. The tasks were followed by discussion and then by the implementation of new tasks, so that the participants could actively incorporate what had been discussed into the new tasks.

Another assessment followed the 30-h training session. This assessment measured those characteristics that we aimed to develop. Again, it took place in the same settlements as the initial assessment. Then, for a year, the participants received constant support from a mentor. The mentor’s job was to provide counselling or any assistance regarding writing CV-s, helping in job searching or preparing for interviews. They did not provide further skill training. A year after the training, a further assessment was carried out to see if any long-term progress could be identified.

In this study, we focus on the results of the assessment and the changes in the specific abilities, the presentation of the whole process is beyond the scope of this paper.

2.3 Method

To measure cognitive abilities, a digitalized test system (PractiWork®, https://practiwork.hu/ 2022) was used. The test system was developed in accordance with strict psychometrical procedures, and it was designed to measure abilities, skills and personality traits. The ability tests were developed on the basis of the CHC theory and therefore each test was designed to measure a chosen narrow ability. Restrictions had to be made on the range of characteristics assessed. As the participants in the programme were volunteers, a measurement protocol had to be designed that would result in them wanting to remain in the programme. For this reason, a balance had to be struck between the amount of characteristics to be assessed and the test time available for completion, so that a sufficiently wide and relevant range of job-relevant characteristics could be assessed, but without the test time being prohibitively long. The completion of the test took about 2.5–3 h.

Based on the research provided by Grobelny (2018) and labor market experience, the system was designed to measure the following abilities:

  • Quantitative reasoning. The ability to perform basic mathematical operations.

  • Deductive reasoning: The ability to understand presented rules and patterns, and to use them to solve tasks.

  • Short-term memory: The ability to apprehend and hold information in immediate awareness and then use it within a few seconds.

  • Perceptual accuracy: The ability to accurately perceive presented stimuli, and select those that complies with the rule.

  • Speed of reaction: The ability to provide a quick reaction to a presented stimulus.

  • Visual debugging: The ability to accurately perceive presented stimuli, and select those that does not comply with the rule.

  • Dexterity: The ability to work precisely with small objects using fine motor skills.

  • Eye-hand coordination: The ability to coordinate eye and hand movements

  • Hand stability: Ability to move the hand and arm in a stable manner

The system uses T-score, a value that is common in psychometrics. T scores in psychometric testing are positive, with a mean of 50, and a standard deviation of 10. T scores represents the number of standard deviations from the mean, most people prefer it because the lack of negative numbers, which means they are easier to work with and there is a larger range so decimals are almost eliminated.

2.4 Results

In this section we present the results of the assessment of the participants. The following Table 1 summarizes the results of the initial assessment.

Table 1. Comparison of the results of reference group and the participants after the initial assessment
Full size table

We found significant differences between the two groups in quantitative reasoning (t (238) = 2.367, p = 0.01), deductive reasoning (t (238) = 3.259, p < 0.01), short-term memory (t (238) = 5.634, p < 0.01), speed or reaction (t (238) = 3.836, p < 0.01), perceptual accuracy (t (238) = 7.167, p < 0.01), dexterity (t (238) = 4.308, p < 0.01), eye-hand coordination (t (238) = 3.900, p < 0.01) and hand stability (t (238) = 2.151, p < 0.01).

The results show that there is a gap in attention skills and motor skills. Therefore, based on the results, we saw it as the most important to design a training programme that targets these areas. The 30-h training, which took place over 5 consecutive days, put the participants in task situations in which they had to use their motor skills, with the aim of working attentively and effectively. The training tasks included, for example, paper folding, but also the use of different hand tools. After the training, another, shorter assessment was conducted. Table 2 shows the results.

Table 2. Changes in the measured abilities after the training
Full size table

We found significant improvement in every measured ability (deductive reasoning: t (238) = 3.259, p < 0.01; visual debugging: t (238) = 5.550, p < 0.01; dexterity: t (238) = 4.484, p = 0.01; eye-hand coordination: t (238) = 2.556, p = 0.01 and hand stability: t (238) = 4.472, p = 0.01). More importantly, the differences we discovered between the reference group and the participants minimized, we could not find statistically significant differences between them in the trained abilities.

Finally, we present the results of the ability assessment that was conducted one year after the initial assessments. The aim of this final assessment was to discover if there are long-term improvements in the trained abilities. Table 3 presents the results.

Table 3. Results of the final assessment one year after the beginning of the programme
Full size table

The results show that a slight decline can be observed in the examined abilities between the after training results and the after one year results. However, the most important result is that the initial differences between the reference group and the participants reduced, and in the case of fine motoric skills, they disappeared (dexterity: t (238) = 0.221, p = 0.824; eye-hand coordination: t (238) = 0.814, p = 0.41 and hand stability: t (238) = 0.65, p = 0.045)). Table 4 shows these results.

Table 4. Comparison of the reference group and the participants one year after the training
Full size table

3 Discussion

Educating people in disadvantaged regions is a major challenge. People living in disadvantaged areas are trying to enter the labor market from a much more disadvantaged position and face a significant digital technology gap. As the results showed, they underperform compared to those who have been already working. It is important to note that this does not mean that their cognitive skills are not sufficient to enable them to enter the labor market. It is possible that people who are not working currently don’t use their skills that are necessary for the labor market, but when it is required, their inactive skills activate, and they can perform well. However, it seems that without help they will perform less well in a selection process, which in turn will hurt their chances in the labour market.

In order to successfully help these people in their post-school education and to orient them towards the right career path, there needs to be a measurement tool, preferably digitally based, with an appropriate theoretical background, to accurately track the level of the individuals being tested.

Our study presented one such practice, where a skills assessment was used to identify areas for improvement (compared to the level of skills on the labor market), to identify training needs and to monitor the level of progress. However, there are some limitations of this work. First, there were serious constraints about what to include in the measurement. The system we used was capable of measuring more cognitive abilities than we actually measured, and if they had been measured, it would have provided further significant information. Second, it would have been interesting to see whether the reference group has changed over a year. Unfortunately, it was not possible to repeat the measurement among them. The lack of a thorough follow-up is also a limitation of our study, however, the pandemic prevented us from carrying out the planned follow-ups.

Summarizing the results, it can be seen that the training has been effective. The manual skills of the participants improved, and, in general, the profile of the trainees matches that of the workers who have been employed for a longer period of time and who form the reference group. These results show that it is quite possible, with the combination of measurement and training based on the results, to improve the abilities of disadvantaged people, thus improving their chances of finding work.