1 Introduction: Why another personality test?

The needs for psychological assessment in organizational contexts are different from the needs that are felt in clinical and educational fields (see for example Sartori and Ceschi 2013; Sartori et al. 2013), even when it comes to personality inquiry (Sartori 2010). For example, personality measures for the assessment of candidates or employees should be at least related to job performance (Rothmann and Coetzer 2003; van der Linden et al. 2010). Even strong correlations between personality tests and job performance might not be sufficient though, because some tests make use of a language people working in organizations are not accustomed to and therefore the personality profiles obtained by those tests may sound meaningless or even abstruse to organizational managers and decision makers (Hogan et al. 1996; Sartori and Rolandi 2013). In general, valid and reliable tests need to be based on empirically supported theoretical models (Sartori and Pasini 2007). In the specific case of personality tests for organizational contexts, they also need to be appropriately developed according to a procedure leading to identify those traits involved in successful performance and express them in an understandable language for users in organizations (Sartori et al. 2014).

A fundamental question in personality research is how many basic dimensions are needed to describe individual differences in personality and, consequently, how many facets are needed to describe different professional profiles in organizations (Holland 1966; Rothmann and Coetzer 2003; van der Linden et al. 2010; Soto et al. 2011). Over the past decades, researchers have made substantial progress in answering this question by using hierarchical models that group behavioral measures into higher-order clusters. One well-known example of such hierarchical models is the Five-Factor Model (FFM, also referred to as the Big Five model, Goldberg 1981, 1990; McCrae and Costa 1999), consisting of such personality traits as Neuroticism, Extraversion, Openness to Experience (from which the well-known acronym NEO), Agreeableness and Conscientiousness. These basic factors have shown to be able to explain and predict individual differences over a wide range of settings, including job performance (Barrick and Mount 1991; Rothmann and Coetzer 2003). In addition, the Big Five have shown to be relevant to different cultures (McCrae and Costa 1997; McCrae, Terracciano and 79 members of the Personality Profiles of Cultures Project, 2005; De Fruyt et al. 2004) and have been recovered consistently in factor analyses of peer- and self-ratings of trait descriptors involving diverse conditions, samples, and factor extraction and rotation methods (Costa and McCrae 1988; Grucza and Goldberg 2007).

FLORA (Sartori 2014) is the name of a new Italian personality test expressly developed for the assessment of specific professional profiles in organizations and based on the FFM. The idea of FLORA has come up with the consideration that Italy lacks a personality test that is both specifically designed for organizations and based on the FFM. The Big Five Questionnaire (BFQ–Capraraet al. 1993) and the Big Five Questionnaire 2 (BFQ 2–Capraraet al. 2007), the Italian psychometric tests based on the FFM, were developed neither to be specifically used in organizations nor for the assessment of specific professional profiles. On the other hand, the Big Five Observer (BFO–Capraraet al. 1994) and the Big Five Adjectives (BFA–Barbaranelli et al. 2002), both based on the FFM and developed for such organizational procedures as assessment center and personnel selection, are mainly instruments for self- and hetero-evaluation, not psychometric tests. Finally, all these instruments only tend to measure the Big Five for themselves, not all the facets that is possible to detect, for example, with the different versions of the NEO-PI developed by Costa, McCrae and Colleagues or in the WAVE (a personality test developed by an international psychometric assessment business which is based on the FFM and is described as a “personality questionnaire for predicting performance and potential”).

In this context, we have developed FLORA, an Italian psychometric test based on the FFM which expressly aims at assessing personality in specific professional profiles described by numerous facets. To do so, and given the specific characteristics that the test was supposed to have, we split the process of its construction and validation into two steps:

  1. 1.

    One qualitative (test development): interviews to employees (in order to detect the personal characteristics involved in successful performance), literature review (in order to organize the characteristics previously detected according to the FFM), theoretical construction (development at desk of the first version of the test);

  2. 2.

    One quantitative (validation process): administration of the first version of the test to a validation sample and, after changes due to exploratory statistical analyses, to a confirmation sample for confirmatory statistical analyses, monitoring of concurrent validity and calculation of the correlations between FLORA and job performance.

2 Test development

2.1 The interviews to employees

Starting from the Critical Incident Technique by Flanagan (1954), the Behavioral Event Interview (BEI) and the STAR (Situation, Task, Action, Result) model, an interview guide was built which was composed of the following 8 questions:

  1. 1.

    What tasks and/or activities is your job made of?

  2. 2.

    What kind of goals are you expected to achieve in your job? Which personal characteristics are useful for the purpose? Which ones are not useful?

  3. 3.

    What kind of difficulties may you encounter in your job? Which personal characteristics are useful in order to cope with them? Which ones are not useful?

  4. 4.

    Which personal characteristics do you think a person should have in order to perform your job at best?

  5. 5.

    Which personal characteristics do you think a person should not have in order to perform your job at best?

  6. 6.

    Can you tell me about an episode of your working life when you found yourself to face a particularly difficult or critical event, and even tell me what you did to cope with it?

  7. 7.

    Can you tell me about an episode of your working life which was of particular success and satisfaction for you, and even tell me how you achieved that kind of result?

  8. 8.

    Can you tell me about an episode of your working life when you did not feel effective in your professional role, and what you feel you have learned from then on?

Thirty-two interviews with 16 different job profiles were carried out. Two work and organizational psychologists were involved for each interview, one as a conductor, the other one as an assistant taking notes. Each interview was audio-registered. Audio-registrations and notes were given to other five work and organizational psychologists who worked together for the extrapolation of the personal characteristics emerged in interviews and the categorization of the personal characteristics according to the Big Five (for further details on the procedure, see Barrick and Mount 1991, pp. 8–9). Such characteristics as abilities, capabilities, skills, competences, aptitudes and attitudes were eliminated in order to keep personality traits only (78 % out of all the characteristics emerged). As for these ones, synonyms and antonyms referring to the same characteristic were unified under one label chosen according to literature. The personality traits not related to the Big Five, such as the ones referring to the Honesty-Humility dimension of the HEXACO model (Ashton and Lee 2007), were eliminated. Content analyses of interviews led to the identification of 28 different personality traits involved in successful performance.

2.2 Literature review and theoretical construction

Referring to the FFM and analyzing such personality tests as the BFQs (Big Five Questionnaires), the 16PF (Sixteen Personality Factors) scales by Cattell and the different versions of the NEO-PI, the 28 personality traits were distributed into the following 5 categories:

  1. 1.

    Extraversion 8 dimensions (activism; autonomy; influence; initiative; interactivity; leadership; multitasking; velocity);

  2. 2.

    Sociability 6 dimensions (care; collaboration; communicativeness; interpersonal sensitivity; positive affectivity; supportiveness);

  3. 3.

    Conscientiousness 5 dimensions (accomplishment; constancy; deliberateness; precision; reliability);

  4. 4.

    Openness 5 dimensions (curiosity; deepening; flexibility; inventiveness; learning);

  5. 5.

    Emotionality 4 dimensions (rmergency management; frustration tolerance; self-control; stress tolerance).

Each dimension was named and operationally defined according to both literature and the organizational aims of the test (Sartori 2014). For each of the 28 dimensions, 6 items were generated, 3 positively and 3 negatively worded. So, in the end, 168 items were created. Another 8 items, drawn from literature and aimed at measuring social desirability (Crowne and Marlowe 1960; Manganelli Rattazzi et al. 2000), were added to form a Lie Scale. All the 176 items were randomized and accompanied by a 7-point rating scale with the following steps: 1 = totally disagree; 2 = strongly disagree; 3 = tend to disagree; 4 = neither agree nor disagree; 5 = tend to agree; 6 = strongly agree; 7 = totally agree.

3 Validation process

3.1 Administration and database construction

3.1.1 Validation sample

In order to test the factor structure of FLORA, a validation sample was used. It was composed of 407 employees, 175 (43 %) males, 232 (57 %) females, aged between 17 and 61 (mean = 38.58, standard deviation = 12.43; mean males = 40.19, standard deviation males = 11.41; mean females = 37.38, standard deviation females = 13.03), with different roles and functions. A subjects-by-items response matrix was built in order to set the data for statistical analyses (basically, exploratory factor analyses).

3.1.2 Confirmation sample

Once a sufficiently stable and robust factor structure was obtained based on the statistical indexes computed on the validation sample, a confirmation sample was used in order to test the factor solutions found with the validation sample. It was composed of 418 employees, 158 (37.8 %) males, 260 (62.2 %) females, aged between 17 and 61 (mean = 38.62, standard deviation = 11.76; mean males = 39.32, standard deviation males = 11.20; mean females = 38.20, standard deviation females = 12.10), with different roles and functions, similar to those of the validation sample. A subjects-by-items response matrix was built in order to set the data for statistical analyses (basically, confirmatory factor analyses).

3.2 Statistical analyses

As for the exploratory analyses, principal factor analyses (PFA) and principal component analyses (PCA) with the criterion of Eigenvalue > 1 and different rotation methods (oblique and orthogonal) were carried out in order to explore the latent structure underlying the items and to monitor construct validity (factor loading cut-off = .30; Cronbach and Meehl 1955; Kline 1993, 1998). Exploratory analyses were carried out by means of IBM SPSS Statistics 19.

As for the confirmatory analyses, starting from the factor solutions obtained in the case of exploration, structural equation models with maximum likelihood method were carried out, in order to test the robustness of the factor models previously identified. Confirmatory analyses were carried out by means of Amos Graphics 18.

Analyses were carried out for each Big Five separately (Extraversion, Sociability, Conscientiousness, Openness and Emotionality) and, within each Big Five, for each dimension of FLORA. In addition, the items belonging to the Lie Scale were analyzed and Pearson r correlation indexes (r) and coefficients of determination (r 2) were computed between each dimension of FLORA and the Lie Scale total score in order to test whether and how each dimension is affected by social desirability (Sartori 2005).

Second-order factor analyses (PFA and PCA) were carried out to test whether FLORA’s dimensions would be grouped in such a way as to reproduce the Big Five model from which the test itself has been generated.

In conjunction with these analyses of validity, Cronbach Alpha coefficients were calculated as reliability measures in terms of internal consistency between items (for the acceptable values the acceptable of Alpha, see De Vellis 2003; Sartori 2004).

Finally, as for concurrent validity, FLORA was administered together with another Italian test named PARI (Prova di Accertamento dei Requisiti di Idoneità; for the characteristics of this test, see Sartori et al. 2014). In addition, FLORA was related to the job performance of 220 trade agents.

4 Results: first-order analyses

Before running PFA and PCA, and in order to test for statistical assumptions, three indexes were computed (Kline 1993, 1998):

  1. 1.

    The subject-to-item ratio;

  2. 2.

    The KMO index (from the name of the authors Kaiser, Meyer and Olkin) of sampling adequacy;

  3. 3.

    Bartlett’s test of sphericity.

The subject-to-item ratio helps to understand how many subjects there are for each variable included in the analysis. A ratio greater than 5 is considered acceptable. The KMO index helps to understand if the sample size is adequate in relation to the number of variables included in the analysis. It varies from 0 to 1 and is considered adequate when it is higher than .60. Bartlett’s test of sphericity tests the hypothesis that the correlation matrix between the variables included in the analysis is an identify matrix, which would mean that all diagonal elements are 1 and all off-diagonal elements are 0, implying that all variables are uncorrelated. If Bartlett’s test of sphericity is statistically significant, then the correlation matrix is not an identify matrix and the set of variables can be analyzed by both PFA and PCA.

4.1 Extraversion

The 48 items belonging to the 8 dimensions of Extraversion were analyzed by a progressive series of PFA and PCA which led to the 7-dimension solution reported in Table 1.

Table 1 Factor solution for the items belonging to the dimensions generated by Extraversion
Full size table

The item-to-subject ratio is 407/48 ≈ 8.5. The KMO index is .79. Bartlett’s test of sphericity is statistically significant for p < .001 (Chi Square Approximation = 8644.247, df = 1128). Ten items from Activism and Velocity are blended together (2 items are out of the model) to form the dimension named Activism, while Leadership and Autonomy are represented by 4 items each. So, after analyses, the items belonging to Extraversion are 42. The total explained variance is 61 %. Cronbach Alpha coefficients are more than acceptable. Confirmatory analyses by structural equation modeling (N = 418) support this solution (SRMR = .05; RMSEA = .04; NFI = .96; CFI = .97).Footnote 1

4.2 Sociability

The 36 items belonging to the 6 dimensions of Sociability were analyzed by a progressive series of PFA and PCA which led to the 5-dimension solution reported in Table 2.

Table 2 Factor solution for the items belonging to the dimensions generated by Sociability
Full size table

The item-to-subject ratio is 407/36 ≈ 11.3. The KMO index is .81. Bartlett’s test of sphericity is statistically significant for p < .001 (Chi Square Approximation = 4921.718, df = 630). Communicativeness is out of the model. Care, Supportiveness and Positive affectivity are represented by 5 items each. So, after analyses, the items belonging to Sociability are 27. The total explained variance is 50.7 %. Cronbach Alpha coefficients are at least acceptable. Confirmatory analyses by structural equation modeling (N = 418) support this solution (SRMR = .04; RMSEA = .04; NFI = .97; CFI = .97).

4.3 Conscientiousness

The 30 items belonging to the 5 dimensions of Conscientiousness were analyzed by a progressive series of PFA and PCA which led to the 5-dimension solution reported in Table 3.

Table 3 Factor solution for the items belonging to the dimensions generated by Conscientiousness
Full size table

The item-to-subject ratio is 407/30 ≈ 13.6. The KMO index is .82. Bartlett’s test of sphericity is statistically significant for p < .001 (Chi Square Approximation = 3913.757, df = 435). The dimensions generated by Conscientiousness are basically confirmed. Only Precision is represented by 5 items. So, after analyses, the items belonging to Conscientiousness are 29. The total explained variance is 57.3 %. Cronbach Alpha coefficients are at least acceptable. Confirmatory analyses by structural equation modeling (N = 418) support this solution (SRMR = .04; RMSEA = .04; NFI = .97; CFI = .98).

4.4 Openness

The 30 items belonging to the 5 dimensions of Openness were analyzed by a progressive series of PFA and PCA which led to the 4-dimension solution reported in Table 4.

Table 4 Factor solution for the items belonging to the dimensions generated by Openness
Full size table

The item-to-subject ratio is 407/30 ≈ 13.6. The KMO index is .80. Bartlett’s test of sphericity is statistically significant for p < .001 (Chi Square Approximation = 3832.536, df = 435). Learning and Curiosity are blended together in one dimension named Learning. Inventiveness and Flexibility are respectively represented by 4 and 5 items. So, after analyses, the items belonging to Openness are 23. The total explained variance is 49.5 %. Cronbach Alpha coefficients are at least acceptable. Confirmatory analyses by structural equation modeling (N = 418) support this solution (SRMR = .05; RMSEA = .04; NFI = .96; CFI = .97).

4.5 Emotionality

The 24 items belonging to the 4 dimensions of Emotionality were analyzed by a progressive series of PFA and PCA which led to the 3-dimension solution reported in Table 5.

Table 5 Factor solution for the items belonging to the dimensions generated by Emotionality
Full size table

The item-to-subject ratio is 407/24 ≈ 17. The KMO index is .84. Bartlett’s test of sphericity is statistically significant for p < .001 (Chi Square Approximation = 3162.363, df = 276). The 4 dimensions were reduced to 3. Stress tolerance and Emergency management are blended together in one dimension named Stress tolerance. Self-control is represented by 5 items. So, after analyses, the items belonging to Emotionality are 21. The total explained variance is 52.8 %. Cronbach Alpha coefficients are at least acceptable. Confirmatory analyses by structural equation modeling (N = 418) support this solution (SRMR = .05; RMSEA = .04; NFI = .98; CFI = .99).

4.6 Lie scale

The 8 items belonging to the Lie Scale were analyzed by a progressive series of PFA and PCA which led to the 1-dimension solution reported in Table 6.

Table 6 Factor solution for the items belonging to the Lie Scale
Full size table

The item-to-subject ratio is 407/5 ≈ 50.9. The KMO index is .76. Bartlett’s test of sphericity is statistically significant for p < .001 (Chi Square Approximation = 348.723, df = 28). The 8 items form one dimension only, but one item has been discarded since its factor loading is lower than .40 and the elimination of this item makes Cronbach Alpha pass from .75 to .77. So, after analyses, the items belonging to the Lie Scale are 7 not 8. The total explained variance is 43.5 %. Cronbach Alpha is more than acceptable.

5 Results: correlations between FLORA’s 24 dimensions and the total score to the Lie Scale

After reversing the scores to the negatively worded items, the total score to each of the FLORA’s 24 dimensions were computed by summing up the scores to the corresponding items and correlated with the total score to the Lie Scale, in order to test whether and how each dimension is affected by social desirability (Tables 7, 8, 9, 10, 11). Given the large sample size (N = 407 + 418 = 825), which raises the probability of statistically significant correlation coefficients, decision was taken to follow the literature on the subject (Kline 1993, 1998) and to consider negligible correlations lower than .30, even if statistically significant (p < .05). Based on this criterion, Constancy only appears to suffer from social desirability, with a correlation coefficient r = .38 and a consequent coefficient of determination r 2 = .14, which means that 14 % of the variance of the answers to the items of Constancy is due to social desirability. As for all the other dimensions, this percentage varies from a minimum of .00016 % of Self-control (practically 0) to a maximum of .065 % of Collaboration (not even 1 %).

Table 7 Correlations between the dimensions belonging to extraversion and the Lie Scale
Full size table
Table 8 Correlations between the dimensions belonging to Sociability and the Lie Scale
Full size table
Table 9 Correlations between the dimensions belonging to Conscientiousness and the Lie Scale
Full size table
Table 10 Correlations between the dimensions belonging to Openness and the Lie Scale
Full size table
Table 11 Correlations between the dimensions belonging to Emotionality and the Lie Scale
Full size table

6 Results: second-order analyses

In order to empirically test the theoretical FFM that generated FLORA (plus Lie Scale), the 24 dimensions (plus Lie Scale) were analyzed by a progressive series of PFA and PCA which led to the 5-dimension (plus Lie Scale) solution reported in Table 12. The item-to-subject ratio is 407/25 ≈ 16. The KMO index is .88. Bartlett’s test of sphericity is statistically significant for p < .001 (Chi Square Approximation = 5245.321, df = 300). As it is possible to see in Table 12, the 5 dimensions theoretically established according to the Big Five personality traits were found. The total explained variance is 78.2 %. Cronbach Alpha coefficients are more than acceptable.

Table 12 Factor solution for FLORA’s 24 dimensions (plus Lie Scale)
Full size table

Confirmatory analyses by structural equation modeling (N = 418) support this solution (SRMR = .04; RMSEA = .04; NFI = .98; CFI = .99).

7 Results: evidence of concurrent validity

It has not been possible so far to administer FLORA with another personality test based on the FFM. Nevertheless, it was our intention to get some empirical evidence of concurrent validity. To do so, we took advantage of the selection activities for aspiring volunteer rescuers carried out in an Italian health association. In this context, we administered FLORA with a validated test named PARI (Prova di Accertamento dei Requisiti di Idoenità; Sartori et al. 2014) which measures two dimensions, one called Attitude and the other one called Reasoning. Attitude measures such aspects as empathy and emotional stability, so the hypothesis was that it would show positive correlations with the dimensions of FLORA belonging to Sociability and Emotionality. Reasoning measures such aspects as causal attribution and logic (verbal, numerical and abstract), so the hypothesis was that it would show positive correlations with the dimensions of FLORA belonging to Extraversion, Conscientiousness and Openness. FLORA and PARI have been simultaneously administered to 1028 subjects. Correlation coefficients shown in Table 13 are in line with hypotheses.

Table 13 Correlations between the 24 dimensions of FLORA and the 2 dimensions of PARI
Full size table

In addition to this and in line with research by Rothmann and Coetzer (2003) and van der Linden et al. (2010), FLORA was administered to 220 trade agents in order to test the hypothesis of correlations between personality traits and job performance. A cross-sectional survey design was used and job performance was expressed in terms of sales figures. According to both the previously conducted interviews to employees and the profile named Enterprising in the model by Holland (1966), hypotheses were that job performance would show positive correlations with dimensions belonging to Extraversion, Conscientiousness, Openness and Emotionality; negative correlations with dimensions belonging to Sociability. Table 14 only shows statistically significant correlation coefficients. Apart from the dimensions belonging to Emotionality, which show no correlations with job performance, the data reported in Table 14 are in line with hypotheses.

Table 14 Correlations between 6 dimensions of FLORA and the job performance of 220 trade agents expressed in terms of sales figures
Full size table

8 Conclusions, limitations and research perspectives

FLORA is an Italian personality test currently composed of 149 items, 78 of which positively worded, 71 negatively worded.

After analyses, both qualitative and quantitative, it is possible to conclude that the characteristics of FLORA, which was developed starting from interviews to employees, seem to meet the criteria to make it a test based on the Five-Factor Model (FFM) and usable for the assessment of specific professional profiles in organizations. Results of exploratory and confirmatory statistical analyses have revealed good indexes of fit (Tables 1, 2, 3, 4, 5, 12) and the dimensions of FLORA have shown to be:

  • Sufficiently uncorrelated with the Lie Scale measuring social desirability (Tables 7, 8, 9, 10, 11)

  • Correlated according to hypotheses to both the two dimensions of PARI (Table 13) and the job performance of trade agents (Table 14).

Although the qualitative part of the study can be considered a strength and statistical analyses are in line with both construction and hypotheses, FLORA needs to be compared with personality tests based on the FFM and administrated to other job profiles other than trade agents in order to deepen its relationship to job performance.

So far norms have been computed on a total sample of 2366 employees, 1135 (48 %) males, 1231 (52 %) females, aged between 17 and 61 (mean = 39.01, standard deviation = 11.65; mean males = 39.75, standard deviation males = 11.07; mean females = 38.55, standard deviation females = 11.98), divided into 16 professional profiles.