1 Introduction

The intense discrimination and inequalities associated with the caste system in India have made explicit use of affirmative and positive action policy (commonly called the reservation policy) in employment, education, and other spheres with respect to discriminated groups such as Scheduled Castes (SC), Scheduled Tribes (ST), and Other Backward Castes (OBC). Many countries practise affirmative action policy for their public and private sectors. However, in India, this policy is confined to the tiny government sector and the vast private sector remains outside the purview of the reservation policy (Thorat and Newman, 2007). With the narrowing of the public sector and unintended (back-door) de-reservation, there is growing demand for some sort of affirmative action policy in the private sector. The issue has also appeared in the election manifestos of political parties and found approval in the Common Minimum Programme of the central (federal) government. Views have been expressed both in favour of and against reservation in the private sector. Employers’ associations, particularly, have opposed this move.

Many commentators acknowledge the prevalence of caste inequality in rural India, but believe that caste discrimination is much less important in urban India. Others believe that caste discrimination occurs primarily in operative jobs, but not in white-collar positions. This paper focuses on inequality in the regular urban labour market in India, and pays special attention to caste-related income and employment gaps among highly educated employees.

This paper is structured as follows. Section 2 discusses the theoretical insights and economic implications of discrimination. Section 3 reviews studies on caste discrimination in the labour market in India. The sources of data and econometric methodology are discussed in Sections 4 and 5. The empirical results are discussed in Section 6. Section 7 concludes the paper and provides policy implications.

2 Theoretical insights and economic implications of discrimination

Becker’s (1971) theory of discrimination, with testable behavioural implications based on a competitive labour market, is often referred to as the “neo-classical theory of discrimination”. In his theory, the motivation for discrimination is based on a non-pecuniary variable, generally designated as “taste for discrimination” against a group. For example, employers with a taste for discrimination against Blacks feel that the real burden is more than the money wage burden; the dissatisfaction they feel at the presence of Blacks in their firm is an additional burden. In the neo-classical theory, not only employers, but also employees and consumers, discriminate against a group. In India, higher-caste employers discriminate against Harijan (“untouchable”) workers. Consumers discriminate when they dislike purchasing goods and services produced by a group, such as SCs or Blacks. Here, the discriminatory behaviour is not based on any objective criteria like quality or price.

Defending Becker’s theory, Arrow (1972, 1973) defined discrimination in terms of the employer’s perception or reality. For Arrow, employers discriminate not because of their “taste to discriminate” but because of uncertainty. Phelps (1972) presents a similar discussion, though he prefers to call his theory the “statistical theory of discrimination”. Arrow and Phelps developed the theory of discrimination based on the lack of information about job applicants.

There are a few other causes for discriminatory behaviour, like social customs (Akerlof 1976, 1980) and monopsony in the labour market (Madden 1975). According to the social customs view, the phenomenon of discrimination occurs because employers maintain certain social conventions. Akerlof’s theory incorporates the social structure into his model to explain the economic phenomenon of income distribution and resource allocation. Specifically dealing with caste-based discrimination, he assumes that utility depends not only on consumption but also on an individual’s prestige and reputation in the society. Hence, a socially conscious individual would discriminate against the group that prevalent social customs traditionally discriminate against. Birdsall and Sabot (1991) note that an employer’s behaviour based on social customs is indistinguishable from that based on taste or prejudice.

Invariably, these economic theories of labour market discrimination are based on a micro-economic foundation, and centre on explaining causes of discriminatory behaviour. Among the various causes elicited, the most probable are those emanating from the taste of some individual (Becker 1971); uncertainty in the labour market (Arrow 1972; Phelps 1972); social customs (Akerlof 1976); and monopsony in the labour market (Madden 1975).

Only a few of these theories delve further into the effects of discrimination on the economy. Tzannatos’s (1987) general equilibrium model made the first attempt to estimate discrimination and its effects on income. The main aim of his study was to find out what would happen to wages and profits if wage differentials by sex were deliberately changed.

A few economists have tried to grapple with the issue of economic exclusion and discrimination based on caste and untouchability in India. In the social science literature, the concept of social exclusion is defined as “the process through which individuals belonging to some groups are wholly or partially excluded from full participation in the society in which they live” (de Haan 1998, p. 1). In this context, two defining attributes of exclusion are particularly recognised: (a) the multiple aspects of discrimination and societal processes; and (b) institutions implicated in deprivation. Sen (2000) draws a distinction between “unfavourable exclusion”, or situations wherein some people are kept out (at least left out), and “unfavourable inclusion”, or situations wherein some are included (even forcibly) under deeply unfavourable terms. Unfavourable inclusion – particularly if the arrangements are unacceptable, or treatment is unequal – may carry the same adverse effects as unfavourable exclusion. The notion of unfavourable inclusion appears to be quite close to the concept of “market discrimination”, which is related to race and gender, in the mainstream economic literature (Becker 1971). Thus, discrimination manifests a situation that involves exclusion or outright restriction on various forms of market entry, and/or selective inclusion, with unequal and unfavourable treatment in participation in various market transactions.

2.1 Economic implications of discrimination

Why are governments in developed and developing countries concerned about economic discrimination? Is discrimination only an equity issue? Does it also involve economic costs to the society? Are the costs it imposes on society more social and political than economic? The insights of mainstream economic theory indicate that economic discrimination, particularly market discrimination, hampers economic growth, leads to unequal income distribution and deprivation for discriminated groups, and creates potential for inter-group conflict (Birdsall and Sabot 1991). Thus, concern about exclusion and discrimination arises not only because it deepens economic inequality and deprives excluded and discriminated groups but also because it adversely affects economic development. Discrimination also affects productivity as discriminated groups reduce the magnitude of their investment in human capital and, therefore, the return on investment drops (Birdsall and Sabot 1991).

The consequences of discrimination on income distribution are obvious as far as market exclusion involves denying access to employment and payment of fair and equal wages. In the labour market, groups discriminated against lose income because their wages/salaries are lower than other groups with similar productivity. Exclusion and discrimination in access to other markets for income-earning assets – such as agricultural land, capital, and others – also reduce the income-generating capacity of groups discriminated against. Further, pre-market discrimination in access to education and skill development prevents discriminated groups from raising their level of human development and reduces their chances of employment.

Groups are discriminated against in exchange not only in the labour market but also in other markets – like land, capital, products, and social services such as education and housing – and non-market channels. Such discrimination is a failure of the market, and has consequences. A policy implication of the neo-classical theory is that intervention in some form is necessary to correct the consequences of such failure. It is this insight that has induced and justified the adoption of various types of affirmative and positive action policies in favour of discriminated groups in the US and other western countries.

In India, affirmative action, or anti-discrimination, policy was advocated mostly on the consideration of the violation of citizen/human rights, particularly of low-caste untouchables. Ambedkar, who pioneered the affirmative action policy, based most of his arguments on the human rights perspective, and drew largely from theoretical development in political science. Ambedkar developed a general theory of caste as far back as in 1916, but its economic underpinnings were elaborated in detail much later in the 1940s and 1950s (and appeared in print only in the 1990s). The efforts to bring about the policy began in 1919, were formalised in 1931 under the Poona Pact, and finally acquired definite legal shape in 1950. Therefore, the discussion on and debate over affirmative action policy in India has, by and large, been devoid of economic logic or justification. This is in contrast to the large body of literature on the analysis of economic discrimination of race, ethnicity, and gender in the western world (Darity 1997).

Since the main justification for affirmative action policy in the West emanates from mainstream theoretical economics, the discussion here is limited to this branch of writing. Generally, this theoretical strand agrees on three unique underlying principles and customary rules that govern and structure the production, organisation, and distribution under the traditional caste system:

  1. a.

    fixed occupation (property rights) for each member of the caste by birth, and hereditary continuation of such occupation;

  2. b.

    unequal distribution of economic and social rights related to occupation, property, employment, wages, education, etc., among different caste groups; and

  3. c.

    the provision of a strong system of penalties to ensure enforcement of the system.

As labour is a part of the production process, labour market discrimination is obviously a part of the exclusionary process of occupation. At a theoretical level, labour market exclusion and discrimination would be manifested in the

  1. a.

    exclusion or implicit restriction on employment from one caste occupation to another; and

  2. b.

    unfavourable inclusion, that is, access or entry to labour employment in another caste’s occupation, but with unequal treatment in wage payment, and other terms and conditions of work.

Essentially, this would mean unequal and lower wages (lower than market or lower than the wages of other groups with the same productivity level) to workers of discriminated groups, along with unequal working conditions governed by coercive, customary, caste-related norms and obligations (Thorat 2001). In terms of consequences, the economic interpretation of Akerlof (1976), Lal (1988) and Scoville (1991) studies implies negative outcomes of caste-based market discrimination for economic growth and income distribution. Akerlof-Scoville-Lal model would thus argue that, given the segmented and imperfect character of the labour market, the economic efficiency of the caste system would be lower than that posited in the model of a perfectly competitive market, and a second-best alternative to the Pareto optimum.

Ambedkar, however, argued that the efficiency and productivity of labour are adversely affected by a number of other ways, such as by the nature of the customary rules that regulate employment, wages, education, and dignity of labour under the caste system. In his view, the efficiency of labour also suffers severely in another manner. In the caste system, economic pursuit is based not on individual choice, individual sentiment, or preference but on the social status of the parents. Social and individual efficiency require an individual to develop their capacity to the point of competency so that they may choose and make one’s own career. This is nearly absent in the caste system, which violates the principle of individual choice by assigning an individual a task or occupation in advance. Some of these occupations are considered polluting or impure and, therefore, socially degrading; the social stigma of impurity and pollution reduce the social status of the persons engaged in them. Forced into these occupations on account of their caste origin, people do not derive job satisfaction, and are constantly provoked to aversion, ill will, and the desire to evade. The caste system also values “physical” work less than “mental” work, and so the concept of dignity of physical labour is nearly absent in the work ethics of the caste system. Consequently, lack of dignity of labour adversely affects the incentive to work. Thus, in view of the standard mainstream theories of discrimination (and also Ambedkar 1936), judged by the standard criterion of economic efficiency, the caste system as an economic organisation lacks all those elements or assumptions required to fulfil the conditions for an optimal economic outcome.

Besides the general negative impact on income distribution, another negative impact of labour immobility across occupations is the social stigma associated with certain occupations, which has been emphasised by both Ambedkar (1936) and Akerlof (1980). By restricting the mobility of labour across caste occupation and thereby not permitting re-adjustment of employment, caste becomes a direct cause of much of “voluntary unemployment” among high-caste persons and “involuntary unemployment” among low-caste persons. The high-caste Hindu would generally prefer to be voluntarily unemployed for some time than to take up an occupation not assigned to his caste. On the other hand, for the low-caste untouchables, the restrictions on taking up another caste’s occupation compel them to remain involuntarily unemployed. Insights from economic theories indicate that market discrimination is a typical case of market failure, as it causes a great deal of economic inefficiency and adversely affects prospects for economic growth. Besides, it entails unequal opportunities to discriminated groups, all of which jointly create a situation of high deprivation and poverty, particularly among low-caste untouchables.

3 Empirical evidence of discrimination against scheduled castes in the private sector: a brief review of past studies

The corporate sector particularly argues against the reservation policy on the grounds that the sector follows fair methods of employment, and hence disregards the need for any anti-discrimination measures. The argument that there is no discrimination in employment in the private sector is completely contrary to the evidence from studies on the working of Indian industrial labour markets (Papola, 2005). In fact, there is considerable evidence to show that the private sector follows exclusionary and discriminatory recruitment methods.

Throughout the period of modern industrial development, Papola observes that the various modes and mechanisms of employment practised by the private sector amply demonstrate the presence of social exclusion and discrimination. During the initial period of modern industrial development, the factory enterprise made use of the jobber system. It was replaced in the 1970s by the institution of the labour contractor/officer. Both systems were exclusionary and biased in nature and outcome. The National Employment Service (NES) Scheme, which followed, is considered less inequitable and discriminatory, but its use by industry has declined over time. Over 1949–53, 50–85 per cent of vacancies notified by employers were filled by those who had registered with the employment exchange. This figure dropped to 65 per cent over 1953–60, and stayed at around 60 per cent over 1960–68. During the 1980s, the ratio was steady at around 55 per cent. Studies reveal that private factories and enterprises do not use the NES in any significant way in recruitment. Papola (2005) reviews some of these studies. Among workers surveyed at different centres at different times, jobs were found through the employment exchange by about 2.2 per cent of the candidates in Pune (1957), 1.87 per cent in Ahmedabad (1971–72), 1.5 per cent in Mumbai (1975–76), and 10.6 per cent in Coimbatore (1986–87). Thus, the only institutional mechanism for ensuring a fair and non-discriminatory process of recruitment has not found favour with private employers; instead, they use informal channels of recruitment on a large-scale basis. The studies alluded to earlier also highlight the percentages of those who found jobs through informal and personalised channels. Such persons comprised more than 70 per cent of the working population in Pune (1959), 60 per cent in Mumbai (1976), and an equally high proportion in Ahmedabad (1975), Coimbatore (1986–87), and Surat (1998). Therefore, it is evident that a very high percentage of workers found jobs through personalised and insider-based recruitment processes. Papola (2005) presumes that these insider groups and persons are socially better endowed. For instance, higher castes, or Brahmins/Marathas, are over-represented in Pune factories –they account for 50 per cent of the workforce but comprise only 35 per cent of the population. In Coimbatore, 49 per cent of Brahmins held protected jobs as against corresponding figures of 23 per cent among OBCs and 30 per cent among SCs.

The study based on field survey of urban areas by Banerjee and Knight (1985) observed that, “There is indeed discrimination by caste, particularly job discrimination—discrimination is the greatest in operative jobs, in which contacts are more important for recruitment, compared with white-collar jobs in which recruitment involves formal methods.” Do Indian labour markets continue to discriminate against SCs/STs, especially in recruiting workers in regular wage and salaried jobs? Based on the NSSO data for 2011-12, the IHD (2014) report on India Labour and Employment shows that in case of upper-caste Hindus, share of regular employment is 31.7 per cent and their share in workforce is only 19.4 per cent. In contrast to this, STs and SCs have much lower share in regular employment in comparison to their share in the workforce. The share of STs in regular employment is 5.0 per cent and their share in workforce is 10.2 per cent; likewise the share of SCs in regular employment is 16.5 per cent and their share in workforce is 19.3 per cent.

Studies suggest that while discrimination is quite significant, a major part of the exclusion is accounted for by endowment (Madheswaran and Attewell 2007), that is, the education, skills, and experience of a worker. Therefore, in the first instance, it is necessary to take effective measures to improve the endowment of workers from disadvantaged social groups. These measures may include not only the provision of support for education and skill formation but also steps towards poverty alleviation. In addition, it is imperative to improve their access to education, training, and health facilities as well as sources of livelihood. This is because it is not sufficient merely to make available institutions and facilities; it is equally important to enable people to avail such opportunities in so far as their existing economic and social handicaps may prevent them from doing so.

However, it has, been observed that endowment is a necessary but not sufficient condition for benefiting from participation in the labour market and access to jobs that befit one’s qualifications. Partially, this exclusion is due to the dissemination of information on jobs is often exclusionary: information becomes available only to those who have access to someone “inside”; and, mostly, insiders happen to belong to groups that are socially and economically better placed than others. This leads to a violation of the equality of opportunity. The discriminatory process can extend beyond access to information to processes of selection wherein attributes that have little relevance for the performance of the job, but tend to favour candidates with better social and economic endowment (for example, the ability to speak fluent English), are emphasised. Therfore, the second necessary condition for reducing exclusion and discrimination is to ensure equality of opportunity in access to information and the use of non-discriminatory methods and criteria in selection.

Finally, it has also been observed that ensuring capacity enhancement and equality of opportunity do not necessarily lead to non-discriminatory and non-exclusionary treatment in labour markets, as employers often have a “taste for discrimination”. To the extent that such discrimination is found to be systematic and significant, the necessary measures lie in the sphere of affirmative action – more specifically, in the form of positive discrimination. Such action in the form of quota and reservation in public sector jobs has yielded significant positive results in India for several decades now, and the private sector may have to consider its adoption in the broader social context as part of its social responsibility towards the disadvantaged sections of society.

In the first major correspondence study in India, Thorat et al. (2007a,b) sent out identical resumes to private companies, both domestic companies and multinational corporations (MNC) in response to newspaper advertisements in New Delhi during 2005–06. The only difference in the resumes was the easily identifiable names of applicants: Hindu upper-caste, Hindu Dalit, and Muslims. The study revealed significant differences between call-backs to the Hindu upper-castes and the other two categories. These findings are confirmed by Siddique (2008) in a study of Chennai, who tests additionally for the interaction between caste and gender, and finds that the lowest call-backs are received by Dalit women.

There are studies of hiring practices which emphasise the role of networks and that of informal and personalised recruitment, where “who you know” is often more important than “what you know”. In a college-to-work study, which tried to uncover the exact pathways through which discrimination manifests itself, Deshpande and Newman (2007) tracked a group of students from the three premier Indian universities in Delhi for two years to understand what jobs they got, how they got them, and what their interview experiences were. It turned out that employers were extremely conscious of the social identity of the applicant, all the while professing deep allegiance only to the “merit” of the candidate. In an employer attitude survey, Jodhka and Newman (2007) find that employers, including MNCs, universally use the language of merit, but managers are blind to the unequal playing field which produces “merit”. Commitment to merit is voiced alongside convictions that merit is distributed by caste and region.

A first-ever caste census of India Inc.’s human resources has revealed that the proportion of SC and ST employees in the private sector in some of the most industrialised states hardly reflects their strength in the general population of those states. The only exception is Tamil Nadu, the top-ranking state in industrialisation and employment (by the number of factories and persons, according to the Annual Survey of Industries 2008-09), where SCs and STs account for almost 18 per cent of the industrial workforce and 20 per cent of the state’s population.

In sharp contrast are some of the other most industrialised states such as Maharashtra, Gujarat, Karnataka, Madhya Pradesh, Rajasthan, and West Bengal, which show a sharp mismatch between the proportion of SCs and STs in the total workforce in the private sector and their proportion in the state’s total population. For instance, SCs and STs constitute 19.1 per cent of Maharashtra’s population, but their share in private sector human resources is only 5 per cent. In Gujarat and Karnataka, SCs and STs comprise just about 9 per cent of the staff strength but, respectively, 22 per cent and 23 per cent of the state’s population (Table 1).

Table 1 SC/ST participation in industry as a percentage of the total population
Full size table

In view of this unambiguous evidence of discrimination, affirmative action becomes essential to guarantee representation to Dalits in preferred positions. It should be noted, however, that due to the specific forms it takes in India, affirmative action cannot be a complete remedy for discrimination. This is because it is applicable only to the public sector, whereas the evidence of discrimination is overwhelmingly from the private sector, which is becoming increasingly important in the Indian economy.

In the first empirical study of the effects on the labour market, Deshpande and Weisskopf (2014) focus on the Indian railways to assess if affirmative action – that is, the presence of SC/ST employees who have gained entry through quotas – has impacted productivity negatively. The study analyses an extensive data set on the operations of one of the largest employers in the public sector in India, and finds no evidence that increasing the proportion of SC and ST employees adversely impacts productivity or productivity growth, as claimed by critics of affirmative action.

4 Sources of data

The present study uses unit-level data collected by the NSSO, India. The employment and unemployment surveys are conducted during 1993–94 (July 1993 to June 1994), 2004–05 (July 2004 to June 2005), and 2011–12 (July 2011 to June 2012). These quinquennial rounds are referred to, respectively, as the 50th round, 61st round, and 68th round. The survey provides data relating to human capital, demographic, and job characteristics of workers. Human capital characteristics include age and education; demographic characteristics include gender, social group, religion, marital status, location (rural/urban), and region; and data relating to job characteristics include industry, occupation, sector, and nature of employment.

The nominal daily wages are deflated by 2001 prices using the official state-level monthly consumer price indices of agricultural labour (base year 1960) for rural wages and consumer price indices of industrial workers (base year 1982) for urban wages (Labour Bureau, various years). The consumer price index data are available for the states like Andhra Pradesh, Assam, Bihar, Gujarat, Haryana, Himachal Pradesh, Jammu and Kashmir, Karnataka, Kerala, Madhya Pradesh, Maharashtra, Orissa, Punjab, Rajasthan, Tamil Nadu, Tripura, Uttar Pradesh and West Bengal. Our analysis used the sample of SC and others/forward caste wage workers in the 15–65 age group in the above mentioned 18 major states of regular urban India.

5 Econometric methodology

In prior research, one finds three different empirical approaches for studying caste discrimination. The first predicts earnings from the characteristics of all workers while including caste as a predictor (a single-equation technique). Unfortunately, this approach yields a biased result, because it assumes that the wage structure is the same for both non-SCs/FCs and SCs). We have employed the following decomposition methods to estimate the extent of discrimination against SC workers in the public and private sector of the regular urban labour market in India.

5.1 Blinder-Oaxaca (1973) Decomposition Method

This decomposition method enables one to separate the wage differential between SCs and FCs into differences that can be explained by differences in characteristics and those that cannot be explained by differences in characteristics. It can be further applied within the framework of the semi-logarithmic earnings equations (Mincer 1974) and estimated via OLS such that

$$\ln \bar{Y}_{Nsc} = \sum {\hat{\beta }_{Nsc} } \bar{X}_{Nsc} + \varepsilon_{Nsc} \left( {{\text{non-SC}}\;{\text{wage}}\;{\text{Equation}}} \right)$$
(1)
$$\ln \bar{Y}_{sc} = \sum {\hat{\beta }_{sc} } \bar{X}_{sc} + \varepsilon_{sc} \left( {{\text{SC}}\;{\text{wage}}\;{\text{Equation}}} \right)$$
(2)

where, \(\ln \bar{Y}\)denotes the geometric mean of earnings, \(\bar{X}\) the vector of mean values of the regressors, \(\hat{\beta }\) the vector of coefficients and \(\varepsilon\)is the error term. Within this framework, the gross differential in logarithmic term is given by

$$\ln (G + 1) = \ln (\bar{Y}_{Nsc} /\bar{Y}_{sc} ) = \ln \bar{Y}_{Nsc} - \ln \bar{Y}_{sc}$$
$$= \sum {\hat{\beta }_{Nsc} } \bar{X}_{Nsc} - \sum {\hat{\beta }_{sc} } \bar{X}_{sc}$$
(3)

The Oaxaca decomposition simply shows that Equation (3) can be expanded. In other words, the difference of the coefficients of the two earnings functions is taken as a priori evidence of discrimination. If, for the given endowment, SC individuals are paid according to the non-SC wage structure in the absence of discrimination, then the hypothetical SC earnings function would be given as

$$\ln \bar{Y}_{sc} = \sum {\hat{\beta }_{Nsc} } \bar{X}_{sc}$$
(4)

Substituting Equation (4) in Equation (3), we get

$$\ln \bar{Y}_{Nsc} - \ln \bar{Y}_{sc} = \sum {\hat{\beta }_{Nsc} (\bar{X}_{Nsc} - \bar{X}_{sc} )} + \sum {\bar{X}_{sc} (\hat{\beta }_{Nsc} - \hat{\beta }_{sc} )}$$
(5)

Alternatively, the decomposition can also be done as

$$\ln \bar{Y}_{Nsc} - \ln \bar{Y}_{sc} = \sum {\hat{\beta }_{sc} (\bar{X}_{Nsc} - \bar{X}_{sc} )} + \sum {\bar{X}_{Nsc} (\hat{\beta }_{Nsc} - \hat{\beta }_{sc} )}$$
(6)

In Equations (5) and (6) above, on the right hand side (RHS), the first term can be interpreted as endowment differences. The second term in these equations has been regarded in the literature as the discrimination component. Studies use either of these alternative decomposition forms (Equation 5 or 6) based on their assumptions about the wage structure that would prevail in the absence of discrimination. This kind of problem is called “the index number problem”.

5.2 Cotton, Neumark and Oaxaca/Ransom decomposition method

To solve the index number problem, Cotton (1988), Neumark (1988), and Oaxaca and Ransom (1994) propose an alternative decomposition. The true non-discriminatory wage would lie somewhere between the NSC and SC wage structure. The Cotton logarithmic wage differential is written as

$$\ln \bar{Y}_{Nsc} - \ln \bar{Y}_{sc} = \sum {\beta^{*} (\bar{X}_{Nsc} - \bar{X}_{sc} )} + \sum {\bar{X}_{Nsc} } (\hat{\beta }_{Nsc} - \beta^{*} ) + \sum {\bar{X}_{sc} } (\beta^{*} - \hat{\beta }_{sc} )$$
(7)

The first term on the RHS of Equation (7) above is skill differences between SC and non-SC, while the second term represents the relative overpayment to NSC due to favouritism, and the third term refers to the underpayment to SC due to discrimination. The \(\beta^{*}\) is the reward structure that would have occurred in the absence of discrimination. The theory of discrimination provides some guidance in the choice of the non-discriminatory wage structure. In Cotton’s (1988) decomposition, the assumption is operationalised by weighting the NSC and SC wage structures by respective proportions of Non-SC and SC in the labour force. The estimator \(\beta^{*}\) is defined as

$$\beta^{*} = P_{Nsc} \hat{\beta }_{Nsc} + P_{sc} \hat{\beta }_{sc}$$
(8)

where \(P_{Nsc}\)and \(P_{sc}\) are the sample proportions of non-SC and SC populations, and \(\hat{\beta }_{Nsc}\) and \(\hat{\beta }_{sc}\) are the non-SC and SC pay structures respectively. The non-discriminatory or pooled wage structure proposed by Neumark (1988) and Oaxaca and Ransom (1994) is written below:

$$\beta^{*} = \varOmega \hat{\beta }_{Nsc} + (I - \varOmega )\hat{\beta }_{sc}$$
(9)

Where I is the identity matrix, \(\varOmega\) is a weighting matrix, which is specified by

$$\varOmega = (X^{\prime}X)^{ - 1} (X^{\prime}_{Nsc} X_{Nsc} )$$
(10)

Where X is the observation matrix for the pooled sample, X Nsc is the observation matrix for the non-SC sample. The interpretation of \(\varOmega\) as weighting matrix is readily seen by noting that

$$X^{\prime}X = X^{\prime}_{Nsc} X_{Nsc} + X^{\prime}_{sc} X_{sc}$$
(11)

where, X sc is the observation matrix of the SC sample. Given \(\hat{\beta }_{Nsc}\), \(\hat{\beta }_{sc}\) and Equation (9), any assumption about \(\beta^{*}\) reduces to an assumption about\(\varOmega\).

5.3 Expanded Decomposition: Combining Wage and Job Discrimination

The Oaxaca (1973), Cotton (1988), and Neumark (1988) methods can be criticised on the ground that they do not distinguish between wage discrimination and job discrimination. Brown et al. (1980) incorporate a separate model of occupational attainment into their analysis of wage differentials. Banerjee and Knight (1985) use this decomposition by introducing a multinomial logit model, which could estimate both wage and occupational discrimination for migrant labourers in India; they define “occupational discrimination” as “unequal pay for workers with same economic characteristics which results from their being employed in different jobs”. We combine elements from Oaxaca and Ransom (1994) and Brown et al. (1980) to form a more detailed decomposition analysis of occupational and wage discrimination (see detailed derivation in Madheswaran and Attewell 2007). The expanded decomposition is given as follows:

$$\begin{array}{lllll} \ln (G + 1) &= \sum\limits_i {{{\tilde \beta }_{iNsc}}} ({{\bar X}_{iNsc}})({P_{iNsc}} - {{\hat P}_{isc}})\left( {{\rm{job}}\;{\rm{explained}}} \right)\\ & \quad + \sum\limits_i {{{\tilde \beta }_{iNsc}}({{\bar X}_{iNsc}})} ({{\hat P}_{isc}} - {P_{isc}})\left( {{\rm{job}}\;{\rm{discrimination}}} \right)\\ & \quad + \sum\limits_i {{P_{isc}}} \left[ {\tilde \beta _i^*({{\bar X}_{iNsc}} - {{\bar X}_{isc}})} \right]\left( {{\rm{wage}}\;{\rm{explained}}} \right)\\ \left. \begin{array}{lllll} & \quad + \sum\limits_i {{P_{isc}}} \left[ {{{\bar X}_{iNsc}}({{\tilde \beta }_{iNsc}} - \tilde \beta _i^*)} \right]\left( {{\rm{wage}}\;{\rm{overpayment}}\;{\rm{to}}\;{\rm{NSC}}} \right){\rm{ wage}}\\ & \quad + \sum\limits_i {{P_{isc}}} \left[ {{{\bar X}_{isc}}(\tilde \beta _i^* - {{\tilde \beta }_{isc}})} \right]\left( {{\rm{wage}}\;{\rm{underpayment}}\;{\rm{to}}\;{\rm{SC}}} \right){\rm{ discrimination}} \end{array} \right\}\end{array}$$
(12)

5.4 Machado–Mata–Melly Decomposition Method

This method was initially developed by Machado and Mata (2005). It is an extension of the Blinder-Oaxaca decomposition method, in the sense that instead of considering the difference at the mean of the wage distribution, it identifies the sources of wage gap at various quantiles of the wage distribution. The Machado–Mata decomposition is based on the estimation of marginal wage distributions consistent with a conditional distribution estimated by quantile regression. One can perform counterfactual exercises by comparing the marginal distributions implied by different distributions for the covariates. The counterfactual distributions are estimated as:

$$\left\{ {\ln \tilde{Y}_{i}^{cf} = x_{i}^{sc} \hat{\beta }_{{u_{i} }}^{Nsc} } \right\}_{i = 1}^{n} \;{\text{and}}\;\left\{ {\ln \tilde{Y}_{i}^{cf} = x_{i}^{Nsc} \hat{\beta }_{{u_{i} }}^{sc} } \right\}_{i = 1}^{n}$$

The latest version of decomposition was developed by Melly (2006). The estimator of Melly decomposition will be numerically identical to the Machado–Mata decomposition if the number of simulation used in the Machado–Mata procedure goes to infinity. The mean square error in the Melly estimation is less than that in the Machado–Mata estimation. The mean square errors of these two estimates converge only if the simulations in the Machado–Mata estimation become very large. The Melly estimator is also consistent and asymptotically normally distributed. For the \(\theta{th}\) quantile, the wage gap between Non-SCs and SCs can be decomposed into two components, as follows:

$$\mathop Q\limits^{ \wedge }_{Nsc} \left( \theta \right) - \mathop Q\limits^{ \wedge }_{sc} \left( \theta \right) = \underbrace {{\left[ {\mathop Q\limits^{ \wedge }_{Nsc} \left( \theta \right) - \mathop Q\limits^{ \wedge }_{cf} \left( \theta \right)} \right]}}_{\text{Effects\,of\,Characterstics}} + \underbrace {{\left[ {\mathop Q\limits^{ \wedge }_{cf} \left( \theta \right) - \mathop Q\limits^{ \wedge }_{sc} \left( \theta \right)} \right]}}_{\text{Effects\,of\,Coefficients}}$$
(13)

Where, \(\mathop Q\limits^{ \wedge }_{Nsc} \left( \theta \right) - \mathop Q\limits^{ \wedge }_{sc} \left( \theta \right)\) is the wage gap estimated from the \(\theta{th}\) quantile of the unconditional log wage distribution for NSCs and SCs respectively; and \(\hat{Q}_{cf} (\theta )\)is the estimated counterfactual unconditional quantile of the log wage distribution for SCs created using the coefficients of non-SCs. It represents the wage distribution of SCs that would have prevailed if SCs had been endowed with their own characteristics but get paid like non-SCs.

The first component on the RHS of Equation (13) is the contribution of the covariates to the difference between wage distributions of non-SCs and SCs in the same \(\theta th\) regression quantile. It measures the wage differential due to the differences in endowment (or characteristics effect). The second component is known as the coefficients effect or discrimination component.

6 Caste discrimination in public and private sectors of regular urban labour market: empirical evidence

6.1 Oaxaca-Blinder Decomposition Results

Initially a single-equation method is adopted, in which wage equations are estimated separately for public and private sector workers with the inclusion of caste dummy in the equations. Due to space constraints, all the econometric results have not been presented in the paper. It is found that in the public sector, SC workers earned 5 per cent less than FC workers in 1993–94, 11 per cent less in 2004–05, and 9 per cent less in 2011–12. Similarly, in the private sector, SC workers earned 11 per cent less than forward caste employees in 1993–94, 17 per cent less in 2004–05, and 18 per cent less in 2011–12. These coefficients are all statistically significant. Thus, the wage gap between SCs and forward castes in the private sector has increased in the post-liberalisation period. The extent of wage gap in public and private sectors has reduced with inclusion of occupation variable in the model. This reduction in wage gap with inclusion of occupation variable in the model implies that discrimination partially operates through occupational segregation, which will be discussed in detail in subsequent sections.

A single-equation approach assumes that the slope coefficients are the same for all social groups. To overcome these limitations, and also since the reservation system – which sets aside a certain proportion of jobs for SC/ST applicants – operates only within the public sector of the Indian economy, we estimated separate earnings functions for the public and private sectors for each social group. Then, we decomposed the earnings differentials between forward castes and SCs for each sector. The descriptive statistics of variables used in the Mincerian earnings function and the regression results are given in Appendix Tables 8, 9 and 10.

From the Blinder-Oaxaca decomposition results given in Table 2, we find that the SC workers are discriminated against in both public and private sectors, but the discrimination effect is smaller in the public sector. The government policy of protective legislation seems to be partly effective. Discrimination still arises in the public sector in part because the reservation quota for lower-caste applicants is close to full only in the less-skilled Class C and D jobs, but is far from filled in the higher category A and B jobs, where higher castes predominate. The evidence provided by these decompositions contradicts the argument that there is no discrimination in the private sector. Claims that discrimination does not occur in the Indian urban private sector are based neither on economic theory nor on empirical facts.

Table 2 Oaxaca-Blinder Decomposition Results for Public & Private Sectors SCs Vs FCs (in Percentage)
Full size table

The large endowment difference, observed in the case of social groups, suggests that pre-labour market discriminatory practices with respect to education, health, and nutrition are more crucial in explaining wage differentials than labour market discrimination (Madheswaran and Attewell 2007). However, it may be noted that the whole part of discrimination component cannot be attributed to current discrimination. It has been argued that unequal labour market outcomes have their roots in discrimination in the past that has caused more harm to deprived backgrounds of the disadvantaged workers. Pre-labour market discrimination affects earnings indirectly by means of lower out of school investments, poor quality of education, field of study, accessibility to higher education, poorer nutrition and health status, and lower social capital. These may result in lower endowments and persistent wage differentials over time (Altonji and Blank 1999; Das and Dutta 2007). Further, the discrimination in access to schooling and to wage employment cannot be controlled for and explained through this analysis.

We also assessed the relative contribution of each independent variable to the observed wage gap. The results given in Table 3 show which part of the wage gap can be attributed to differences in endowments and which to differences in rewards (discrimination) in the earnings function. If we look at the total difference column, the proxy for experience – the age variable – was favourable for FCs in the public sector and for SCs in the private sector. Note that the large contribution of age for SCs in the private sector is more than offset by the constant term, which is in favour of FCs. The education variable is favourable for SCs in the public sector at almost all levels of education except at graduation and beyond but only at the primary and middle levels of education in the private sector. Women are in a disadvantaged situation, as the male dummy is negative and favourable to SCs in both public and private sectors. Being married and in a permanent job is favourable for FCs in both public and private sectors. Finally, the region variable is favourable for FCs in public sector and for SCs in the private sector.

Table 3 Relative Contribution of Specific Variables to Decomposition in Public and Private Sectors of Regular Urban LM: Oaxaca-Blinder Decomposition Method, FCs Vs SCs, 2011–12
Full size table

6.2 Cotton, Neumark, and Oaxaca/Ransom decomposition results

The decomposition results were calculated using the Cotton (1988), Neumark (1988), and Oaxaca and Ransom (1994) approach. Of these three estimates, which one is least objectionable? To answer this question, standard errors were estimated for each. The pooled method (Oaxaca and Ransom) has a smaller standard error, and should probably be preferred. When this method is used, the discrimination coefficient is somewhat smaller in magnitude, but there is still clear and substantial evidence of discrimination in the labour market against SCs.

Table 4 shows that the wage difference due to skill is 80.6 per cent in the public sector and 68.3 per cent in the private sector. This skill or productivity advantage is estimated as it would have been in the absence of discrimination. The FC treatment advantage (benefit of being in the labour market) is 5.3 per cent in the public sector and 8.1 per cent in the private sector. This is the difference in wages between what FCs receive currently and what they would receive in the absence of discrimination. The treatment disadvantage (cost of being in the labour market) component for SCs is about 14.1 per cent in the public and 23.6 per cent in the private sector. This is the difference in the current SC wage and the wage they would receive in the absence of discrimination. This form of the decomposition procedure yields more accurate estimates of the wage differential, but it also models the true state of differential treatment by estimating the “cost” to the group discriminated against as well as the “benefits” accruing to the favoured group. The cost of being SCs in the labour market is very high –they are hugely underpaid.

Table 4 Cotton-Neumark –Oaxaca/Ransom Approach- FCs Vs SCs: Public and Private Sector of Regular Urban LM, 2011–12 (In Percentage)
Full size table

6.3 Estimating Occupational Segregation, Wage, and Job Discrimination Against Scheduled Castes in Public and Private Sectors

In the previous analysis, we found that with inclusion of occupation variables in the earnings equation, the final calculation of the discrimination coefficient was reduced at least by 10.2 per cent in the public sector and by 4.7 per cent in the private sector. It implies that discrimination operates partially through occupational segregation. This result motivated the authors to incorporate occupational attainment in the decomposition; estimate job discrimination against SCs in labour market; compare the actual and predicted occupational distributions of SCs and FCs in public and private sectors in the regular urban labour market; and analyse occupational attainment equation within the framework of a multinomial logit model. Using the occupational attainment results, a predicted occupational distribution for SC \((\hat{P}_{sc} )\) was obtained. For non-SCs, this estimation procedure yields a predicted distribution identical to their actual sample distribution, i.e. \(\hat{P}_{Nsc} = P_{Nsc}\). The difference in the predicted distributions, \((P^{Nsc} - \hat{P}^{sc} )\), is the “explained” component due to differences in characteristics. The residual difference, \((\hat{P}^{sc} - P^{sc} )\), is the ‘unexplained’ component due to differential access or discrimination. It is found from Tables 5 and 6 that except in administrative, professional, and clerical occupations, the residual component accounts for the major part of the observed differences in both public and private sectors; in professions related to services, sales, and production, the residual difference is positive even if the observed difference is negative. It implies that if there were no differential access to occupations by caste, the proportion of SCs in occupations related to service, sales, and production would be higher than of non-SCs.

Table 5 Full Decomposition of Gross Earnings Difference between SC and FC Workers in Public Sector of Regular Urban LM: 2011–12
Full size table
Table 6 Full Decomposition of Gross Earnings Difference between SC and Forward Caste Workers in Private Sector of Regular Urban LM: 2011–12
Full size table

For estimating wage and job discrimination simultaneously, the earnings functions have been estimated separately for each occupation in the public and private sector. The findings from the expanded decomposition analysis show that in the public sector, job discrimination against SCs is more pronounced than wage discrimination in all categories of occupations, except in elementary occupation. In the private sector, job discrimination against SCs is more pronounced than wage discrimination in all categories of occupations except in clerical and elementary occupation. The magnitude of job discrimination is higher in the private sector than in the public sector except in clerical-, production-, and trade-related occupations. The wage discrimination component is decomposed into wage overpayment to FCs and wage underpayment to SCs. We found that the treatment disadvantage (cost of being SCs in the labour market) component for SCs is higher than the treatment advantage of FCs irrespective of type of occupation and sector except in elementary occupation of the public sector.

In conclusion, discrimination in labour market accounts for a large part of the gross earnings difference, with occupational discrimination (inequality in access to certain occupations) being considerably more important than wage discrimination (unequal pay within a given occupation, given one’s educational and skill level) in both public and private sectors of India. This result is consistent with the findings in Madheswaran (2011), which supports the argument that job discrimination against SCs occurs in white-collar jobs, recruitment to which involves more formal methods and in the operative jobs, in which contracts are more important for recruitment. The prevalence of caste discrimination in the public sector despite the government’s reservation policy shows its inefficiency.

6.4 Wage gap across wage distribution quantiles

The authors try to decompose the wage gap between FCs and SCs within the public and private sectors at different quantiles of the wage distribution. The results of the Machado–Mata–Melly decomposition given in Table 7 show that the contribution of endowment difference to the raw wage gap is higher in the public sector than in the private sector irrespective of wage distribution quantile. The unexplained part (discrimination) of the wage gap reported in Figure 1 is lower within the public sector than private sector, except in the lower wage distribution; indeed, the public sector is always below the private sector across the entire wage distribution. The unexplained part of the wage gap decreases within the public sector when we move along the wage distribution, whereas the opposite is true for the private sector. Irrespective of the methodology we use, there is clear empirical evidence, which indicates that the degree of discrimination against the disadvantaged group is very high in the private sector.

Table 7 Sector-wise MMM Decomposition Results across Quantiles: Others Vs SC-Regular Urban LM, 2011–12
Full size table
Fig. 1
figure 1

Discrimination Coefficients across the Quantiles in Public and Private Sector of Regular Urban LM, 2011-12

Full size image

7 Concluding observations and policy implications

While other countries used various affirmative action policies in both public and private sectors from the very beginning, India confined such a policy only to the minuscule public sector and excluded the vast private sector. Against this background, different rounds of National Sample Survey data are used in order to examine the wage gap between FC and SC workers in the regular urban labour market. The main conclusions based on decomposition methodology are listed below.

  1. a.

    Due to discrimination, wages are lower for SCs than equally qualified FCs by 19.4 per cent in the public sector and by 31.7 per cent in the private sector, but the discrimination effect is much larger in the private sector.

  2. b.

    The cost (underpayment) of being SC in the private sector labour market is 23.6 per cent compared to 14.1 per cent in the public sector; and

  3. c.

    The contribution of endowment difference to gross wage differential is larger than the discrimination coefficient. It has been argued that unequal labour market outcomes have their roots in discrimination in the past that has caused more harm to deprived backgrounds of the disadvantaged workers. Pre-labour market discrimination affects earnings indirectly, by means of lower out-of-school investment, poor quality of education, field of study, accessibility to higher education, poorer nutrition and health status, and lower social capital.

  4. d.

    Discrimination accounts for a large part of the gross earnings difference between the two caste groups in the public and private sector of regular urban labour market, with occupational discrimination (unequal access to jobs) being considerably more important than wage discrimination (unequal pay in the same job).

  5. e.

    The quantile regression results reveal that there is a “glass ceiling effect” in the private sector and a “sticky-floor effect” in the public sector. The unexplained part of the wage gap decreases within the public sector when we move along the wage distribution, whereas the opposite is true for the private sector.

Irrespective of the methodology we use, there is clear empirical evidence that the degree of discrimination in the private sector against disadvantaged groups is very high. Therefore, this paper provides strong evidence for the extension of affirmative action policy in the private sector in India, and for the federal government to enact an “equal opportunity law” to provide legal safeguards against discrimination.

In the light of empirical results, this paper recommends the following policies.

The government provides private sector units safeguards to promote their business and trade, and designs its foreign and export–import policy to help businesses set up by individuals. Foreign investors also invest in the private sector by purchasing shares, which is made possible by government policy. The upliftment of the weaker sections is a stated objective of our country, and thus the implementation of reservation in the private sector is part of the social responsibility of both the government and the private sector. In fact, it is merely the fulfillment of the agenda of distributive justice enshrined in various articles and clauses of the Constitution of India. Even though the private sector uses public money via public financial institutions, it does not enforce reservation for SCs, STs, and OBCs. That is the reason for the demand of reservation in the private sector. If the private sector is not fulfilling its social responsibility, the government should make such provisions through legislative measures.

While the discursive debate about providing reservation in the private sector is on, there are some concerned citizens who are calling for systematic planning and enforcement of some measures that would contribute both to nation building and improving the lives of marginalised communities. It is suggested that an Employment Opportunity Commission be constituted to review and ensure that weaker sections find representation at all levels. Special provisions should be made for higher education, responsive training, and multi-skilling of tribals and SCs, so that they are able to compete with FCs for jobs. The National Commission for Scheduled Castes and the National Commission for Scheduled Tribes should be empowered so that they can work as pressure groups and pressurise the government and the private sector to promote the right to participatory development. Finally, a nationwide debate should be held on these issues and the necessary constitutional amendments should be introduced to enact affirmative action at all levels in the private sector.