1 Introduction

Cultural diversity in the workplace has become a topic of great interest to organizational researchers and practitioners alike. Studies have shown that organizations would do well to include employees of different cultural backgrounds in their workforce, as this may enhance productivity (e.g. Cox et al. 1991; Jehn et al. 1999; McLeod and Lobel 1992; Van Knippenberg et al. 2004). However, successfully managing diversity appears to be difficult, as it is also associated with negative team outcomes (Jehn et al. 1999; Mannix and Neale 2005; Van Knippenberg and Schippers 2007) and resistance among employees (Thomas and Plaut 2008; Harrison et al. 2006; Antwi-Boasiako 2008).

In order to effectively implement diversity policies, it is essential to first understand employees’ attitudes towards diversification. It has become apparent that diversity attitudes and/or perceptions of employees may be a strong predictor of diversity outcomes, such as social identification patterns, team cohesion, employee well-being and performance (Hofhuis et al. 2012; McKay et al. 2007; Van Knippenberg et al. 2013). The perceptions of the majority group in particular may have a strong influence on the effectiveness of diversity policies that are implemented within the organization.

In this paper we describe the development and validation of the Benefits and Threats of Diversity Scale (BTDS), an instrument which measures employees’ perceptions towards the effects of diversity. The BTDS presents enhanced utility with regard to similar instruments in that it (1) measures perceived positive and negative effects of diversity on independent subscales, and (2) provides more specific and concrete dimensions of both positive and negative effects than any existing measures, thus increasing its practical applicability.

This paper closely follows the flow of our research. We first provide a literature review of potential benefits and threats of diversity in the workplace, which forms the conceptual framework on which the BTDS is based. In study 1, using qualitative data gathered in interviews, this conceptual framework is tested against perceptions of managers in the field. Based on the results from study 1, the theoretical framework is enhanced to more accurately reflect the actual diversity perceptions which are found in practice.

In study 2, the development of the individual items of the BTDS, based on both existing literature and the qualitative data gathered in the interviews, is presented. The internal structure, reliability and construct validity of the BTDS are established using quantitative data gathered in existing organizations.

1.1 Importance of measuring perceptions towards diversity

Research on cultural diversity often focuses on the experiences of minority groups within a multicultural context. Consider, for instance, research based on Berry’s (1997) acculturation strategies, which has provided extensive knowledge on psychological processes which influence immigrant groups’ position within a society. One should not forget, however, that interaction between cultural groups is by definition a two-way-street. Attitudes and behavior of the majority group may also strongly influence those of minority members. In this vein, Bourhis et al. (1997) have shown that, in addition to acculturation strategies employed by immigrants, the host society’s preference for immigrant’s acculturation strategies may be just as influential on the outcomes of the acculturation process.

Within an organizational context, the same processes may apply. Empirical studies have shown that the majority’s attitude towards diversity is a strong predictor of minority members’ sense of acceptance, and that a positive attitude is beneficial for intergroup contact (Hofhuis et al. 2012; Tropp and Bianchi 2006; Tropp et al. 2006).

Also, like any type of organizational change, implementing new diversity policies may cause resistance (Thomas and Plaut 2008). Harrison et al. (2006) provide a meta-analysis showing how affirmative action programs, some of the most widely used diversity interventions, may have negative impact on majority members’ acceptance of diversity. They also mention, however, that these effects may be reduced by the way diversity policies are communicated and justified within the organization. As such, we believe that in order to overcome diversity resistance among majority members it is of crucial importance for the organization to be aware of existing diversity perceptions within its employee base and use this information to shape its diversity management strategies.

In earlier research, cultural diversity in the workplace has been related to both positive and negative outcomes. Employees, then, may perceive the positive effects of having a diverse workforce, but may also feel threatened by its perceived negative effects. Studies have shown that these perceptions of diversity in turn influence (positively or negatively) the actual outcomes of diversity (e.g. Cox and Blake 1991; De Meuse and Hostager 2001; Hostager and De Meuse 2008; Schneider and Northcraft 1999; Van der Zee et al. 2009).

1.2 Reasons for developing the BTDS

Several instruments have been developed which measure general attitudes towards diversity in the workplace. For example, Hostager and De Meuse (2008) developed the Reaction-to-Diversity (R-T-D) Inventory which can be used to categorize respondents into diversity optimists, realists or pessimists. Similarly, the Attitudes towards Diversity at Work Scale (ADWS; Nakui et al. 2011) was constructed to measure employees’ attitudes to workplace diversity on two components: productive (beliefs of effective productivity of diverse workgroups) and affective (social or affective aspects of diversity). Both are examples of instruments that provide a reliable and valid assessment of majority attitudes on diversity. However, in order to conduct successful diversity management, organizations may need more detailed information.

The BTDS has two major advantages over existing instruments. Firstly, it measures positive and negative attitudes towards diversity on separate dimensions. As proposed by Van Knippenberg and Schippers (2007) in their review of diversity literature, we believe the inherent ambivalence of diversity outcomes may not be fully represented in one scale ranging from pro to con (see also Stockdale and Cao 2004). As diversity may lead to different positive and negative outcomes simultaneously, employees may also perceive both sides of the coin at the same time. As will become apparent, researchers should be aware of this potential ambiguity.

Secondly, the BTDS enables researchers to compile a ranking of different types of concrete benefits or threats as perceived by employees, thus providing much more detailed information to its users. One of the major strengths of this instrument is that both its typology of benefits and threats, as well as the individual items used to operationalize these constructs, are based on existing literature and then augmented with actual perceptions of employees gathered in interviews. The instrument and its subdimensions are then validated using a large scale survey study. The combination of qualitative and quantitative data used in the development of the BTDS increases its external validity and ensures a higher degree of practical applicability.

As a starting point we will review existing literature explaining concrete benefits and threats of diversity which may be perceived in the workplace.

1.3 Benefits associated with cultural diversity

Why do organizations aim to increase cultural diversity in their workforce? A review of the literature suggests that diversification may be perceived as beneficial for the overall productivity of an organization in several different ways.

Firstly, it is recognized that an organization’s markets and stakeholders are inherently culturally diverse. Hence, having a diverse workforce is a valuable tool for gaining knowledge about, and access to minority groups within society (e.g. Ely and Thomas 2001). An example is a supermarket in a culturally diverse neighborhood, which matches the cultural backgrounds of its employees with those of its customers to provide the best customer service. The same principle can be applied to other types of organizations; for example, a governmental organization must have a diverse workforce in order to understand and meet the needs of different groups within the society it serves. Understanding of diverse groups in society may thus be seen as the first benefit of cultural diversity.

Secondly, in addition to added knowledge about external stakeholders (target groups, clients, customers), earlier research has also provided evidence that diverse teams may be more innovative and creative in completing their assigned tasks (Van Knippenberg et al. 2004). Studies show that diversity reduces groupthink and may lead to more extensive, and more original idea generation (Fay et al. 2006; Nakui et al. 2011; Nijstad and De Dreu 2002; West 2002). De Dreu and West (2001) argue that when the opportunity is given to voice different viewpoints (cf. Edmondson 1999), the presence of deviant opinions may increase creative thinking and force team members to be more alert and critical in their evaluation of problem solving strategies (Brodbeck and Greitemeyer 2000; Collins and Geutzkow 1964). According to Ely and Thomas, the ‘insights, skills and experiences employees have developed as members of various cultural identity groups are potentially valuable resources that the work group can use to rethink its primary tasks and redefine its markets, products, strategies, and business practices in ways that will advance its mission’ (Ely and Thomas 2001, p. 240). In sum, cultural diversity may lead to higher flexibility, creativity and, ultimately, higher effectiveness of the organization as a whole (Shipton et al. 2005). Increasing the creative potential within organizations may thus be seen as the second dimension of benefits of diversity.

Finally, in addition to the direct positive effects on productivity, cultural diversity may often be seen by organizations as a means to generate a positive image towards the outside world. By showing off the diversity of its workforce, an organization implies it is a socially responsible institution, providing equal opportunities to all cultural groups and aiming to reduce discrimination (cf. Cunningham and Melton 2011). The idea that organizations may aim to increase diversity out of moral obligation ties in to recent literature on corporate social responsibility and business ethics in modern organizations (e.g. Bear et al. 2010; Bird et al. 2007). We propose that the positive effect on an organization’s image of social responsibility forms the third dimension of benefits of cultural diversity.

1.4 Threats associated with cultural diversity

An often cited framework for analyzing the different types of threat that may be experienced in a multicultural context is Stephan and Stephan’s Integrated Threat Theory (ITT; 2000), which deals primarily with the attitudes of majority groups towards minorities. ITT names three types of threat that may be experienced in a culturally diverse environment: realistic threats (external circumstances that involve potential physical, economic or status loss for the in-group), symbolic threats (the perception of the out-group’s beliefs, values and symbols as a threat to the in-group’s beliefs, values and symbols), and intergroup anxiety (negative feelings of in-group members when anticipating or experiencing contact with out-group members).Footnote 1

In the workplace, realistic threats of diversity include anticipated negative effects for one’s career, status, power or influence. Increased attention for minority members’ career prospects, for example through affirmative action programs, will inherently reduce career chances for majority employees at the same level (Antwi-Boasiako 2008). This may be particularly relevant in times of reorganization or downsizing of companies, which already reduces employees’ sense of security, even without taking cultural diversity into account. Earlier studies have indeed shown that realistic threat is related to lower support for diversity policies among majority employees (Lowery et al. 2006).

A second dimension of diversity-related threat in the workplace is employees’ dislike of having to change familiar behavior, and to be confronted with a worldview that potentially does not fit with their own. Termed symbolic threat by Stephan and Stephan (2000), this threat is a result of (perceived) differences in norms, values, beliefs and attitudes between cultural groups. The realization that one’s own worldview—as a function of one’s cultural heritage—is only relative makes the world less predictable and less comprehensible, which in turn induces a sense of fear (Greenberg et al. 1990). Symbolic threats can be theoretically explained through the existence of different social identities and subsequent categorization processes that emerge in diverse social environments (for a review, see Van Knippenberg et al. 2004). Within organizations, symbolic threat particularly manifests itself as a resistance to cultural change (Thomas and Plaut 2008). An organizational culture often reflects the culture of the majority group. As a result of increased diversity, established norms and values may change to incorporate some of the minority’s cultural background, thus inducing symbolic threat in majority employees.

Thirdly, intergroup anxiety is defined as a negative feeling of in-group members when anticipating or experiencing contact with out-group members (Curseu et al. 2007). Several studies have shown that this form of anxiety may lead to exaggerated cognitive, affective and behavioral reactions towards diversity (see Staw et al. 1981). For a review of the difficulties associated with intercultural interaction and its theoretical underpinnings, see Van der Zee and Van Oudenhoven (2000). In the workplace, intergroup anxiety may manifest itself as a reluctance of employees to interact with or give feedback to colleagues with a different cultural background, for fear of making a mistake, or being embarrassed by the interaction.

Finally, as a fourth possible diversity-related threat, we propose to include the potential for loss of team effectiveness. Although diversity may increase productivity under certain conditions (see above), research shows it may also negatively affect team performance (Jehn and Bezrukova 2004; Thomas 1999). For instance, the abovementioned threats show that a diverse team may be more difficult to manage; diversity may reduce social cohesion and increase the risk of miscommunication and conflict. These effects may impede the work process and in turn decrease innovation and creativity (Van der Zee and Paulus 2008; Paulus and Nijstad 2003). Furthermore, prejudice towards minority employees (particularly immigrants) often ascribes them lower language proficiency, or lesser education (Choenni 2007; Curseu et al. 2007). As such, inclusion of minority employees may be perceived as a potential risk to team effectiveness. We therefore argue that perceived Productivity Loss (PL) as a result of diversification should be viewed as a separate dimension of threat.

1.5 Present research

In this paper, we present an instrument which is designed to map detailed diversity attitudes, by providing insight in employees’ perception of several concrete benefits and threats of cultural diversity. The development of this instrument will be described in two studies.

In study 1, we collected perceptions of positive and negative outcomes of diversity among employees in the field, in semi-structured interviews, using critical incident technique (Flanagan 1954). Our aim was to collect as many critical incidents of specific diversity outcomes as possible. Next, we tested these critical incidents against the typology of benefits and threats outlined above. Our goal was to assess whether the perceptions of employees in the field reflected the benefits and threats found in existing research. Where necessary, the framework described in this introduction was extended with findings from the interviews, to generate an exhaustive categorization of perceived benefits and threats that may be prevalent in the workplace. Finally, the critical incidents gathered in the interviews were used to construct individual items for each of these dimensions of benefits and threats, thus ultimately forming the BTDS.

In study 2, we validated the BTDS by examining its psychometric characteristics, including internal structure, subscale-reliability and construct validity. The latter was assessed by exploring the relationship of perceived benefits and threats with several theoretically related constructs. These include two personality measures, Cultural Empathy and Emotional Stability which have been shown to affect employees’ functioning in a multicultural context (Van der Zee and Van Oudenhoven 2000). Also, we will examine the relationship of the BTDS with diversity climate; a measure of the degree to which the organizational climate displays openness towards and appreciation of diversity (Hofhuis et al. 2012; Luijters et al. 2008).

2 Study 1

Study 1 aimed to map the benefits and threats of diversity perceived by managers in the field, using qualitative data gathered in interviews. In order to generate an exhaustive categorization of perceived benefits and threats, we asked the target group to openly reflect on their own experiences and perceptions regarding diversity, without specific prompting by the researchers. This method provided us with a broad spectrum of perceived benefits and threats which are prevalent in the workplace, while ruling out directive questioning or priming of the respondents. The perceptions of managers are subsequently tested against the framework of benefits and threats derived from literature. Where necessary, the conceptual framework established in the introduction was extended with findings from practice. The results of this study were subsequently used to compile the subscales and individual items that make up the BTDS.

2.1 Method

2.1.1 Respondents

Interviews were conducted with 19 employees in several divisions of the Netherlands’ public service. Candidates were recruited through a network of personnel managers in the participating divisions. The researchers did not know the interviewees before contacting them. Respondents were not given any information on the aims of the study, aside from the fact that they would be interviewed about diversity in the workplace. Of those interviewed, 68 % was male, mean age was 47 (range 41–53). All respondents had received higher education.

2.1.2 Procedure

Data were gathered by two researchers using semi-structured interviews. The interview structure was first piloted on two employees working in the same sector as the target group. The piloted interview structure was sufficiently effective, but several small changes in question formulation were made to finalize it. The results from the pilot interviews were not included in the final study.

During the interviews, the critical incident technique (Flanagan 1954) was used to gather examples of concrete work situations in which the respondents experienced or perceived positive or negative effects of cultural diversity. All interviews were conducted in Dutch, which was the primary working language of all respondents.

After demographic questions, respondents were asked to reflect on their perception of cultural diversity, and experienced or expected benefits and threats. These questions were structured as follows: “Can you give me a concrete example of a [positive/negative] development or event related to cultural diversity, which [has happened in the past/is happening presently/you expect may happen in the future]. The interviewers were instructed to keep on asking for more incidents (cf. Flanagan 1954) before moving on. When respondents were unable to recall any more incidents or perceptions, they were presented with three guiding questions, including: “Can you give me any more concrete examples of diversity related developments of events which may influence, positively or negatively, [the way your department completes its assigned tasks/your department’s daily work routine/the way people interact on the work floor]”. It was found that these guiding questions rarely provided new information; most respondents had provided all their views and perceptions in the first set of questions.

2.1.3 Analysis

The interviewers transcribed the critical incidents using tape recordings of the interviews. Next, the interviewers separately compiled all critical incidents into categories, keeping in mind the conceptual framework outlined in the introduction. If an incident could not be placed into the existing framework, a new category was created. The independent categorization judgments of the two interviewers were then compared, revealing agreement on approximately 80 % of the critical incidents. Categorization was then discussed and adjusted until full consensus was reached on which incident belonged to which category.

2.2 Results

In Appendix 1, we provide short definitions for dimensions of perceived benefits and threats of cultural diversity, as extrapolated from the interview data, including an example of two critical incidents (translated into English by the authors) which belong to the respective category.

2.2.1 Benefits

Results (see Appendix 1) show that all three of the dimensions of benefits found in literature (Understanding Diverse Groups in Society, Creative Potential, and Image of Social Responsibility) were represented in the critical incidents gathered in the interviews. This provides support for the relevance of these dimensions for employees in the field, thus confirming the validity of the theoretical concepts.

Furthermore, a number of critical incidents regarding benefits of cultural diversity could not be placed into the existing dimensions. Based on the content of these critical incidents, we were able to identify two distinct categories, both of which were mentioned by a substantial number of respondents. This prompted us to include two additional dimensions of benefits.

Job Market refers to the notion that cultural diversity may be beneficial for an organization’s position regarding recruitment and retention of employees. Many respondents (26 %) mentioned this as an important reason for increasing cultural diversity in their department. Being able to recruit from all cultural groups in society allows them to choose from a larger pool of potential talents. Additionally, several respondents mentioned that, depending on the job market, cultural diversity may be necessary for filling all vacancies with qualified personnel. This concept ties in to earlier research by Fields et al. (2005), which provides evidence that organizations which face problems in recruiting or retaining an adequate supply of human resources, display a higher percentage of minority employees (see also Pfeffer and Salancik 1978). On the basis of our interview findings, we may conclude that this may be the case for the organizations included in our study.

Secondly, 32 % of respondents mentioned that, aside from the other benefits described above, cultural diversity may also have a positive impact on social interactions in the workplace. They implied that the presence of different cultural groups in their department is ‘fun’ and leads to a more inspiring and comfortable work environment. Since nearly one-third of respondents specifically mentioned this, we decided to include Social Environment in our framework as the fifth dimension of benefits. In table 1, the percentages are shown of interviewees who, without specific prompting, mentioned the different dimensions.

Table 1 Number and percentage of interview respondents \((n = 19)\) who mentioned the BTDS dimensions
Full size table

2.2.2 Threats

Regarding perceived threats of cultural diversity, the critical incidents gathered in the interviews reflect the four dimensions in our theoretical framework (see Table 1; Appendix 1). No additional categories were needed, thus confirming the typology of threats presented in the introduction of this paper.

2.3 Discussion

The aim of study 1 was to map perceived benefits and threats of cultural diversity as viewed by employees in the field, and examine whether these were in accordance with dimensions found in literature. Regarding benefits, the theoretical dimensions Understanding Diverse Groups in Society, Creative Potential, and Image of Social Responsibility were matched by the perceptions of respondents. Furthermore, two new dimensions were added, namely Job Market and Social Environment, on the basis of the fact that they were also mentioned by a substantial number of respondents. Particularly the fact that respondents, often mentioned that diversity can lead to a more pleasant social environment is an interesting result, as it is inconsistent with common research findings that show diversity has a negative effect on social cohesion and team identification (e.g. Jehn et al. 1999).

Regarding threats, the four theoretical dimensions, realistic threat, symbolic threat, intergroup anxiety, and productivity loss, were in concordance with the respondents’ perceptions. In total we have thus identified five benefits and four threats of diversity. These dimensions form the underlying structure of the BTDS, which will be described below.

3 Study 2

This study describes the development and validation of the Benefits and Threats of Diversity Scale (BTDS). The items which represent the subscales of the BTDS were constructed by the researchers, using condensed formulations of the critical incidents gathered in study 1, examples of which are provided in Appendix 1. An English translation of the individual items that were included in the BTDS are provided in Appendix 2.

In the present study, we report the psychometric qualities of the BTDS, using data collected in a large scale field study in several public service organizations. First, we tested the internal structure of the a priori subscales, using confirmatory factor analysis. Next, Multiple Group Method (MGM) (Nunnally and Bernstein 1987; Kiers 1990) was used to examine whether the individual items were sufficiently representative of their intended subscales. Additionally we examined the intercorrelations between the different subscales, as well as their respective means and standard deviations.

The construct validity of the instrument was assessed by examining the relationship of the BTDS’ subscales with three theoretically related constructs. Firstly, Cultural Empathy is a personality trait which reflects the degree to which individuals are able to empathize with the feelings, thoughts and behavior of members of different cultures. It has been shown that individuals who display strong cultural empathy view diversity as a challenge rather than a threat, and view cultural differences as interesting and inspiring (Van der Zee and Van Oudenhoven 2000; Van der Zee and Van der Gang 2007). We therefore predict this trait to be positively related to perceived benefits of diversity in the workplace.

Secondly,Emotional Stability has been defined as an individuals’ ability to remain calm in stressful (intercultural) situations and cope effectively with the insecurity which results from intercultural interaction (Van der Zee and Van Oudenhoven 2000). Low emotional stability has been related to higher levels of insecurity and anxiety (e.g. Pickering and Gray 1999) and may lead to a stronger emphasis on the preservation of one’s own culture in multicultural work situations (Arndt et al. 1997). Earlier research has shown that emotional stability may reduce the negative impact of cultural diversity on an individual’s sense of security (Van der Zee et al. 2004). We therefore predict emotional stability to be negatively related to perceived threats of diversity in the workplace.

As a third test of construct validity, we examined the relationship between the BTDS subscales and perceived Diversity Climate - an organizational climate characterized by openness towards and appreciation of diversity (Hofhuis et al. 2012; Luijters et al. 2008). Diversity climate is reflected in the possibility to choose one’s own work style and maintain important cultural habits in the workplace, even though these habits may differ from what is perceived as ‘normal’ (Luijters et al. 2008). Also, a strong diversity climate means the presence of diversity among employees is seen as an advantage, and not a nuisance. It has been established that diversity climate enhances job satisfaction of both majority and minority employees and reduces the negative impact of cultural diversity on social cohesion and team effectiveness (Hofhuis et al. 2012; Van Knippenberg et al. 2004). We therefore predict that diversity climate is positively related to perceived benefits, and negatively related to perceived threats of diversity in the workplace.

3.1 Methods

3.1.1 Procedure and respondents

Data reported in this study were gathered in nine separate divisions within the Netherlands’ public service. They were recruited through the diversity managers within each organization’s personnel division. After agreeing to participate, the diversity managers made a random selection of employees from their organization’s personnel database. The individuals in the selection were contacted through e-mail, inviting them to participate by clicking on a link to a digital questionnaire. Respondents were allowed to fill in the questionnaire during work hours. No compensation was given.

In total, about 5000 respondents were contacted. The overall response rate was approximately 26 %. A total of 1,295 majority employees fully completed the questionnaire, forming the final sample for this study.

In the sample, 57 % were male; average age was 44.3 (range 22–68). Due to the exclusion of support staff, respondents were highly educated (92 % had a college degree). Of the respondents in this sample, 11 % \((n = 143)\) were in a management position. Comparison with existing personnel data (POMO 2007) shows this is a representative sample of employees within the participating sector.

3.1.2 Instruments

The BTDS consists of 36 items formulated in Dutch—four for each of the dimensions (five benefits, four threats)—which measure the degree to which employees feel cultural diversity may lead to the particular benefit or threat. An example of an item measuring Creative Potential is ‘Cultural diversity\({\ldots }\)enables us to come up with more original ideas’. An example of an item measuring Realistic Threat is ‘Cultural Diversity\({\ldots }\) leads to fewer career opportunities for majority members.’ All items are based on the critical incidents gathered in the interviews conducted in study 1. For a full overview of the items, see Appendix 2. In the questionnaire used in this study, the BTDS’ items were mixed randomly.

Cultural Empathy \((\alpha = 0.76)\) and Emotional Stability \((\alpha =0.77)\) were each measured using five items, adapted from the Multicultural Personality Questionnaire (MPQ; Van der Zee and Van Oudenhoven 2000). An example of an item measuring Cultural Empathy is ‘I understand the norms of other cultures’; an example of an item measuring Emotional Stability is ‘I am often nervous’.

Finally, Diversity Climate \((\alpha =0.71)\) was measured using six items, as used by Hofhuis et al. (2012). An example of an item is ‘In this organization, cultural differences between colleagues are openly discussed’.

All constructs were measured on a 5-point scale, ranging from ‘Completely disagree’ (1) to ‘Completely agree’ (5).

3.2 Results

In this section, we examine the psychometric characteristics of the BTDS. First an overview is given of the internal structure and reliability of the instrument. Secondly, we examine the intercorrelations between the BTDS’ subscales, and their respective means and standard deviations. Finally, we assess construct validity using the BTDS’s correlations with Cultural Empathy, Emotional Stability and Diversity Climate.

3.2.1 Internal structure

The internal structure of the BTDS was assessed in two steps. First, a confirmatory factor analysis was conducted to assess whether the a priori subscales were supported by the data, using EQS 6.1 (Bentler 2005). Two models were tested. Model A was a two-factor model in which individual items were related to either benefits (factor 1) or threats (factor 2). Model B was a nested model in which the two main factors are further divided into five and four subscales respectively, which is the intended structure of the BTDS. To assess the degree to which the models fit the variance in the population, a Chi-square test is usually conducted, in which a significant Chi-square is indicative of a poor fit. However, the significance of the Chi-square test is strongly dependent on the sample size and is almost always significant in sample sizes greater than 400 (Tanaka 1987). In addition chi-square is affected by the size of the correlations in the model: the larger the correlations, the poorer the fit. Because our study has a relatively large sample size \((n=1{,}295)\) and a high risk of interfactor correlations due to the close theoretical relations between BTDS subscales, it is more informative to consider other indices to assess model fit. Root Mean Square Error of Approximation (RMSEA) was calculated, using the procedure described by Browne and Cudeck (1993) and Steiger (1990). Lower RMSEA indicates better fit. The prevailing convention is that an RMSEA value smaller than 0.08 is indicative of a good fit (Schermelleh-Engel et al. 2003). The fit of Model A (two factors) is adequate (\(\text {CFI}= 0.99\); \(\text {RMSEA}=0.082\), 90 % CI [.080, 0.083]). However, analysis of Model B, which is the intended structure of the BTDS, displays a respectable increase in fit (\(\text {CFI}= 0.99\); \(\text {RMSEA}= 0.058\), 90 % CI [.057, 0.059]). The RMSEA confidence intervals of the two models do not overlap, meaning Model B is significantly better at explaining variance. These results show that the intended subscales add considerably to the internal validity of the instrument, thus providing support for the intended structure.

In order to test whether the individual items are an accurate representation of their intended subscales, further confirmatory analyses were conducted using MGM (Nunnally and Bernstein 1987; Kiers 1990). MGM was designed to detect possible deviations in the allocation of individual items to subscales. The procedure is based on the analysis of item-rest correlations between each item and the other items in its intended subscale, as well as correlations with the other subscales. Using MGM, the internal structure can be assessed using two indicators (Stuive et al. 2008). Firstly, the total variance explained by the a priori subscales, using MGM, should be comparable to the total variance explained by a Principal Components Analysis (PCA). This indicates the structure of the intended subscales is a good explanation of the original variance in individual items. Secondly, individual items themselves should display a stronger correlation with their intended subscale than with other subscales. The latter can be tested by computing a \(z\)-score (and corresponding \(p\)-value) for the difference between two correlations, using a Fisher transformation (Fisher 1921). When the \(z\)-score is sufficiently high, this means the item loads highest on its own subscale, thus confirming its structural validity. Below, we report our findings regarding these indicators for subscales of benefits and threats separately.

3.2.2 Benefits

Table 2 shows the results of the confirmatory analysis using MGM on all items measuring benefits. A PCA on these 20 items provided a total explained variance of 73.2 %, whereas using MGM, the total variance explained is 68.9 %. Thus, the variance explained between the two methods is comparable, which is an indication that the intended subscales provide a satisfactory explanation of the variance in the original items.

Table 2 Confirmatory analysis of benefits using multiple group method \((n=1{,}295)\)
Full size table

Pairwise comparisons between the correlations in the table, using Fisher transformations (Fisher 1921), reveal that most items display stronger correlations with their own intended subscale than with other subscales (\(p < 0.01\), one-tailed). The only notable exceptions are two items which, in addition to relating strongly to their intended subscale, also display an equally strong correlation with another subscale. The item ‘Cultural diversity in our department\(\ldots \) is fun’, intended to measure Social Environment is also strongly related to the Creative Potential subscale (se03; \(z = 0.72\); \(p = 0.246\)). The item ‘Cultural diversity in our department \({\ldots }\) leads us to have more choices when recruiting and selecting new personnel’ intended to measure Job Market is also strongly related to the Creative Potential subscale (jm03; \(z = 0.91\), \(p = 0.181\)). Although these two items display some overlap with another subscale, they still correlate most strongly with their intended subscale. Therefore, they will be included in further analyses.

Internal reliabilities of the five Benefits-subscales are good (see also Table 5). Cronbach’s Alpha’s are all above 0.80, with the exception of Job Market \((\alpha = 0.78)\), which is still more than satisfactory.

These results largely confirm the internal structure of the BTDS’s dimensions of Benefits, and show that the subscales are sufficiently reliable for further use.

3.2.3 Threats

Table 3 shows the results of the confirmatory analysis using MGM on all items measuring threats. A PCA on these 16 items explains 67.7 % of total variance. A confirmatory analysis using MGM shows that the a priori subscales explain 71.0 % of total variance, which is 3.3 % more than the PCA. This indicates that the expected subscales are a good explanation for the variance in the original items.

Table 3 Confirmatory analysis of threats using MGM \((n=1{,}295)\)
Full size table

Pairwise comparisons reveal that most items correlate more strongly with their intended subscale than with other subscales (\(p < 0.01\), one-tailed). Exceptions are two items in the subscale PL which, aside from correlating with their intended subscale, also display an equally strong correlation with the Intergroup Anxiety subscale. These are ‘Cultural diversity\({\ldots }\) reduces the overall quality of employees’ (pl04; \(z = 2.01\), \(p = 0.023\)) and ‘Cultural diversity\({\ldots }\) makes our department more difficult to manage’ (pl02; \(z = 0.80\); \(p = 0.206\)). Furthermore, the item ‘Cultural diversity\({\ldots }\) leads to a situation in which majority members are forced to adjust’ which is intended to measure Symbolic Threat, is also strongly correlated with the Realistic Threat subscale (st03; \(z = 0.97\); \(p = 0.166\)). Although these three items display some overlap with other subscales, they still correlate most strongly with their intended subscale. Therefore, they will be included in further analyses.

Internal reliabilities of the four subscales are good (see also Table 5). Cronbach’s Alpha’s are all above 0.80, with the exception of Symbolic Threat \((\alpha = 0.77)\), which is still more than satisfactory.

These results largely confirm the internal structure of the BTDS’s dimensions of Threats, and show that the subscales are sufficiently reliable for further use.

3.2.4 Intercorrelations between subscales

Table 4 shows intercorrelations between all 9 subscales of the BTDS, including both benefits and threats. Consistent with the slight overlap found in the MGM-analyses, relatively strong correlations are found within benefits (\(0.48 < r < 0.75\)) and threats (\(0.52 < r < 0.80\)). Respondents who perceive one type of benefit, also perceive others; the same applies to threats. Although these intercorrelations are higher than would be preferred, we are still able to successfully distinguish between the different subscales, as will be shown by subsequent analyses.

Table 4 Correlations between BTDS subscales and theoretically relevant constructs
Full size table

The results generally show no strong correlations between benefits and threats. The two main factors of the BTDS are mostly independent. This suggests higher levels of perceived benefits of diversity do not necessarily go together with lower levels of perceived threats, or vice versa.

There are two exceptions to the above findings which are worth noting. Firstly, the dimension Social Environment displays negative correlations with all dimensions of threats \(( -0.16 > r > -0.22)\). As can be expected, the perception that diversity is fun and inspiring, may act as a buffer against the negative effects of diversification in the workplace. Secondly, the dimension Symbolic Threat, defined as a threat to an individual’s norms and values, is negatively related to Social Environment, and positively related to all other benefits (\(0.25 < r < 0.38\)).

3.2.5 Means and group differences

Table 5 shows the mean and standard deviation for each of the BTDS subscales. Respondents tend to score significantly lower on the scales measuring threats than those measuring benefits. Also, the results reveal subtle, yet significant, gender differences on the mean scores and standard deviations. Female respondents scored significantly higher on benefits and lower on threats than men, which is consistent with earlier studies showing more favorable attitudes towards diversity among women (e.g. Van der Zee et al. 2009). It must be noted that, although significant (\(p <\) .05), the absolute gender differences found in this study are relatively small.

Table 5 Cronbach’s alpha, means, standard deviations
Full size table

Finally, age of respondents displays significant, but relatively weak, positive correlations with Job Market \((r = 0.10)\) and Symbolic Threat \((r = 0.14)\). indicating older respondents score slightly higher on these two subscales. The other dimensions are unrelated to age.

3.2.6 Correlations with multicultural personality and diversity climate

To assess the construct validity of the BTDS, we examined its relationship with three theoretically related constructs. Firstly, Table 4. displays the correlations of the BTDS subscales with two personality measures adapted from the Multicultural Personality Questionnaire (MPQ; Van der Zee and Van Oudenhoven 2000). Cultural Empathy was predicted to show a positive relationship with benefits, which is confirmed by our data (\(0.18 < r < 0.40\)). Individuals who are more able to empathize with the feelings, thoughts and behavior of members of other cultures also perceive more positive effects of diversity in the workplace. Additionally, we find a negative relationship with three of the four threats (\(-0.19 > r > -0.20\)), which shows these individuals also perceive less negative effects of diversity.

As predicted, Emotional Stability is negatively correlated with all threats (\(-0.17 > r > -0.25\)). This indicates that the better individuals are able to cope with uncertainty as a result of intercultural interactions, the less they feel threatened by cultural diversity in the workplace.

Finally, as predicted, the results show that Diversity Climate is positively related to perceived benefits (\(0.11 < r < 0.37\)) and negatively related to perceived threats (\(-0.12 > r > -0.34\)). When the organizational climate is characterized by openness and appreciation of cultural diversity, individuals perceive more positive and less negative effects of diversification.

3.3 Discussion

Overall, the results confirm the internal structure, reliability and construct validity of the BTDS. Using MGM, we have shown the a priori subscales of the instrument are a good explanation for the total variance in individual items. All items correlate strongly with their intended subscale, and reliabilities are high.

Intercorrelations between the subscales show that the dimensions of benefits, as well as the dimensions of threats, are partially related. However, the two main factors of the BTDS are shown to be largely independent. Exceptions are the specific dimensions Social Environment which is also (negatively) related to threats, and Symbolic Threat, which is also related to benefits.

Finally, all BTDS’ subscales are related to multicultural personality and diversity climate in the predicted directions. Cultural Empathy predicts individuals’ perceived benefits of diversity, whereas Emotional Stability is shown to be a buffer for individuals’ perceived threats. Employees who experience a strong Diversity Climate in the workplace, perceive more positive and less negative effects of diversification. These findings support the construct validity of the BTDS.

4 General discussion

This paper describes the development and validation of a new instrument designed to measure employee perceptions of cultural diversity in the workplace, the Benefits and Threats of Diversity Scale (BTDS). In relation to other instruments measuring employees’ overall attitudes towards diversity (e.g. De Meuse and Hostager 2001; Nakui et al. 2011), the BTDS includes several unique features. Firstly, existing instruments often view attitudes towards diversity on only one dimension, from positive to negative. The BTDS distinguishes between perceived positive and negative outcomes of diversity on separate dimensions, which expands on the possible uses of the instrument. Secondly, the BTDS enables researchers to compile a ranking of different types of benefits or threats as perceived by employees. The fact that the BTDS subscales are based on qualitative findings in the field improves its external validity and practical applicability. The knowledge that can be gained by using the BTDS may be essential for reducing diversity-related resistance and increasing the overall effectiveness of diversity management in modern organizations.

4.1 Summary of findings

A conceptual framework was established regarding possible positive and negative effects of cultural diversity in the workplace, based on both existing literature and qualitative data gathered in interviews. The results confirm that existing research findings largely coincide with experiences of employees in the field, but two new dimensions of benefits emerged from the interview data which were not included in the proposed framework. These were consequently added to the proposed instrument.

The final structure of the BTDS includes five dimensions of perceived benefits of cultural diversity in the workplace: Understanding of Diverse Groups in Society, Creative Potential, Image of Social Responsibility, Job Market and Social Environment. The four dimensions of perceived threats of cultural diversity in the workplace are Realistic Threat, Symbolic Threat, Intergroup Anxiety and PL. Using critical incidents gathered in interviews, four items were constructed for each of these dimensions, together forming the subscales of the BTDS.

Study 2 confirmed that the BTDS is sufficiently reliable for further use. However, some improvements could be made to increase the subscales’ individual distinctiveness. Our confirmatory factor analyses revealed a total of five items which display some overlap with another subscale, besides loading strongly on subscale they were intended to measure. For future use, it may be beneficial to slightly alter these items to make them more reflective of the construct measured in their respective dimension.

The reported means and standard deviations show that respondents tend to score in the center of the scale for all subscales, although some slight variations were found; i.e. scores on threats seem to be relatively lower than on benefits. We realize these findings may be dependent on the context of this research—public service organizations in the Netherlands. The public service employees that were included in our research were employed in a wide range of different functions, within a range of different divisions of the public service (including, policy departments, tax consultants, legal advisors, and inspection teams, just to name a few) which increases our confidence in the generalizability. Nonetheless, it is important to further validate the instrument in different types of organizations and other work settings. To minimize the influence of social desirability in the responses to the BTDS, future researchers should be careful with the formulations used when introducing the instrument. It should be clear to respondents that the scales are not designed as a ‘test’ to uncover subtle racism, but instead are intended to measure the overall diversity attitudes within the organization without imposing judgements.

Some group differences in responses were found: women tend to score slightly higher on benefits, and men tend to score slightly higher on threats. These effects are consistent with earlier research (e.g. Van der Zee et al. 2009), showing that women tend to display more positive attitudes towards diversity. Although researchers should account for these effects when using the BTDS, the absolute differences on the scores are very small and may easily be controlled for in further use of the instrument.

The construct validity of the BTDS was assessed using correlations with two personality measures and one measure of organizational climate; constructs which are different in nature from individual perceptions of benefits and treats. Usually, these types of analyses are conducted using constructs which are much more closely related to the actual dimensions of the instrument which is tested, resulting in strong correlation coefficients. Remarkably, even though the constructs used in this study are conceptually further removed from the BTDS’ subscales, we still find all correlations are in the predicted direction, albeit weaker than usually found in validation studies. Although these findings support the construct validity of the BTDS subscales, further research could provide additional evidence by relating the BTDS’ subscales to conceptually closer instruments such as the Reactions-To-Diversity Inventory (De Meuse and Hostager 2001) and the Attitudes towards Diversity at Work Scale (Nakui et al. 2011).

With the above comments in mind, the BTDS is sufficiently reliable to be used by researchers to gain detailed insight in the diversity attitudes of majority employees in modern organizations.

4.2 Theoretical implications

The data presented in study 2, aside from confirming the validity and reliability of the BTDS, also provide several theoretical insights which may have implications for further research. Firstly, the intercorrelations between the different subscales of the BTDS show that the respective dimensions of benefits and threats are mostly independent. This implies that individuals who perceive many benefits of diversity do not perceive less threats, or vice versa. This, in turn, suggests that individuals are not either pro or con diversity, as is often assumed in existing literature, but instead may possess a more nuanced view. Employees in this sample tend to see both positive and negative effects as independent, and are aware that benefits and threats of diversity are not mutually exclusive. Our results confirm, therefore, that it is important to not only measure diversity attitudes on a single dimension, but instead to measure both positive and negative attitudes on separate scales (cf. Van Knippenberg and Schippers 2007). This may reveal a more accurate picture of how individuals really feel towards diversification, and allow for a more complete analysis of the processes involved.

Secondly, although benefits and threats seem to be mostly unrelated, we have identified two exceptions which provide further insight in the manner in which individuals perceive the effects of diversification. The dimension Social Environment, reflecting the degree to which individuals feel diversity leads to a more pleasant work environment, is not only related to other benefits, but also displays a negative relationship with threats. Secondly, Symbolic Threat, the degree to which individuals fear a change of norms and values within the organization, is positively related to perceived benefits of diversity. Earlier reviews of the diversity literature (e.g. Van der Zee and Paulus 2008; Van der Zee et al. 2004) conclude that the positive effects of cultural diversity often involve productive outcomes for the organization, such as creativity and learning potential, whereas the negative effects of diversity are often more affective in nature, involving a sense of threat and insecurity for the individual self. The results of our studies confirm that perceptions of majority employees in organizations towards diversity are inherently ambiguous. The intercorrelations between the BTDS subscales suggest that respondents who perceive many positive effects of diversity may also be aware that diversity leads to a certain degree of change, which in turn induces symbolic threat. Consequently, our results also show that respondents who feel cultural differences are fun and inspiring, at the same time feel less threatened by diversification. Both findings confirm the notion that successful diversity management should aim at finding a balance between social cohesion and psychological security on the one hand, and organizational change and innovation on the other (see also Van der Zee and Paulus 2008).

A logical next question is how to achieve a situation in which employees are both aware of the benefits of diversity, and feel less threatened by its negative effects. The results presented in study 2 provide some evidence that an individuals’ multicultural personality, particularly the traits Cultural Empathy and Emotional Stability, may be beneficial. The former is particularly related the perception of benefits, while the latter may serve as a buffer reducing the negative impact of diversity-related threats. Finally, our results show that in organizations which are characterized by a high degree of openness and appreciation towards diversity, termed a strong Diversity Climate, employees may also perceive more benefits and less threats of diversity. Therefore, creating such an organizational climate may be essential in fostering positive attitudes towards new diversity policies and reducing diversity-related resistance.

4.3 Conclusions and practical implications

In many studies, employees’ attitudes towards diversity have been measured on one dimension, ranging from positive to negative. By developing the BTDS, we have provided organizational researchers with an instrument to study diversity attitudes in much greater detail. Using the BTDS scholars may be able to further unravel the complicated social processes which take place in culturally diverse organizations.

The BTDS also has many direct practical uses for today’s organizations. Oftentimes, the process of diversification causes resistance among majority members in organizations (Thomas and Plaut 2008; Harrison et al. 2006). This resistance in turn makes many diversity interventions ineffective. Imagine, for example, a group of managers who fear that diversity policies will reduced their own career chances. Next, imagine this same group participating in a seminar on cultural sensitivity and open-mindedness. This intervention will probably not be very successful; participants may not want to include minority members in the organization in the first place, let alone be sensitive or open-minded towards them. Understandably, when their perceived threats are not addressed, individuals may be less willing to invest time and energy in reaping the benefits of diversity.

To effectively cope with resistance, it is essential to have detailed insight in employees’ perceptions regarding cultural diversity. The BTDS can be used as an instrument to measure where resistance is likely to originate. By analyzing employees’ perceptions, organizations may be able to communicate more effectively about diversity, and target those employees that feel most threatened by the process of diversification. Secondly, knowledge of the majority group’s perceptions allows organizations to effectively choose interventions which are in line with employees’ needs and expectations. As such, the BTDS may be an essential tool for increasing the effectiveness of diversity management in modern organizations.