1 Introduction

This paper revisits the issue of mortgage-lending discrimination, which has vital implications for egalitarianism in general and individuals’ social and economic welfare in particular. We specifically address two old questions about whether race, gender, and ethnicity affect (1) the probability of banks granting a new mortgage loan and/or (2) the spread that banks charge on newly originated loans. We also ask a new question: What is the role of the subprime crisis with respect to these issues? The main novelty of our work, relative to the extant literature, rests in improvement of identifying discrimination practices when using countrywide U.S. loan-level data, along with placing the role of the subprime crisis at the heart of our analysis. Answering these questions in the most precise way possible is especially relevant for policy-makers and the general public, because housing is, inter alia, a core part of human well-being and a well-known part of the American dream (Pager and Shepherd 2008).

Existing literature on identifying discriminatory practices in the mortgage loan market considers both discrimination in a bank’s decision to originate the loan or not (henceforth called loan origination discrimination) and discriminatory practices in the pricing of loans (henceforth called loan pricing discrimination). The premise is that minority applicants, mostly African-Americans or Hispanics, are potentially discriminated against relative to non-minority applicants (i.e., whites and non-Hispanics) with respect to loan origination and/or pricing. The most prominent branch of the existing literature augments information from the Home Mortgage Disclosure Act (HMDA) with information on specific loan applicants and property values, etc., and it shows that white applicants have at least an 8% higher chance of receiving a loan relative to African-Americans, ceteris paribus (e.g., Munnell et al. 1996; Hubbard et al. 2012; Wheeler and Olson 2015).

Identifying discriminatory practices was one important reason for the creation of the HMDA database. Unfortunately, the HMDA does not provide crucial information on several dimensions of the bank’s decision-making process regarding loan origination and pricing. The missing information includes, but is not limited to, the loan-to-value ratio as well as the applicant’s wealth and creditworthiness. Thus, inference on discriminatory practices from research that uses only the HMDA database will very likely overstate the level of discrimination significantly and thus be invalid. For this reason, researchers and policy-makers tend to examine specific subsets of the HMDA—usually in local markets, where information on these basic omitted variables is available.

This approach can also be criticized, however, along two dimensions. First, the list of important omitted variables can be very idiosyncratic at the individual level. In fact, when discriminatory practices are alleged the involved financial institutions defend their operating methods based on this argument (see the numerous lawsuit cases filed by the National Association for the Advancement of Colored People 2007Footnote 1). Essentially, this idiosyncratic omitted-variable problem means that any augmentation of an HDMA subset might still lack the information required to clear the endogeneity of the variables characterizing, for instance, race or ethnicity. Second, even in the unlikely event that researchers and policy-makers had access to augmented local data, and these data were sufficient to overcome the omitted-variable bias for these local markets, inference across the full gamut of mortgage loans and the entire United States cannot be made. As a result, regulators find discriminatory practices very hard to identify, let alone penalize and reduce. In addition, new legislation on discrimination might be hampered by the lack of specific evidence.

This paper’s main goal is to introduce a new empirical strategy that significantly reduces the omitted-variable bias in identifying mortgage-loan discrimination (both in loan origination and in loan pricing) using information solely from the HMDA, and thus across the entire United States, for a very large number of loan applications (more than 6 million from 2004 through 2013).

Our method runs in two stages. In the first stage, we estimate separate equations for the requested (i.e., the amount in the loan application) and granted (i.e., the amount in the loan application if the loan is originated; zero if the loan is not originated) loan amounts, with regional (census-tract) fixed effects, year fixed effects and, in some sensitivity tests, individuals’ income as the explanatory variables. Subsequently, we take the difference of the residuals between the two equations, which includes the unobserved reasons behind banks’ willingness to originate a loan or not relative to the applicant’s reasons in applying for the loan. Among these unobserved reasons is also the potential effect of race and gender. In the second stage, we estimate a binary model on the bank’s decision whether or not to grant the loan, using observable characteristics such as race, ethnicity, and gender as independent variables along with the difference in the residuals from the first stage. Essentially, this means that we extract the effects of race, ethnicity, and gender from the remaining unobserved component that is also included in the binary model.

Our results indicate that white applicants had approximately a 0.39% higher probability of being granted a loan than African-Americans from 2004 through 2013, and non-Hispanic applicants had a 0.25% higher probability than Hispanic applicants. These estimates, although statistically significant, are quite low and show limited discrimination on average. Further, our estimates are significantly lower compared with, for example, most previous studies’ estimates of 8% or higher loan approval for whites and non-Hispanics. Our findings support the suggestion that omitted-variable bias is very idiosyncratic and cannot be easily captured, even in data sets augmented with additional applicant characteristics. The equivalent estimates for gender are even lower and economically unimportant.

In forming our two-stage model, we discuss the potential bias from imprecise estimation of the first stage, as well as bias from applicants’ income misreporting. We carry out a number of sensitivity tests to remedy these sources of measurement error, including the use of the model of Erickson et al. (2014) and of data only from regions with low probability of income misreporting (Piskorski et al. 2015).

Moreover, by splitting our sample to the periods before and after 2007, we find an economically small, albeit notable, increase in the estimated level of racial and ethnic discrimination after 2007. Specifically, before the crisis, white (non-Hispanic) applicants have a 0.23% (0.14%) higher probability of being granted a loan compared with African American (Hispanic) applicants, whereas after the crisis, the foregoing probability increases to 0.66% with respect to African Americans and 0.54% with respect to Hispanics. Therefore, we find that the subprime crisis increased discrimination, even though these estimates remain far lower than those of previous studies.

Subsequently, we turn to discrimination in loan pricing. We show that, when all other factors are equal, white applicants pay 0.37 basis points lower spreads on their mortgage loans compared with African Americans and non-Hispanic applicants pay 0.11 basis points lower spreads compared with Hispanics. The difference between genders is negligible. The significant difference in spreads between minority and non-minority applicants is observed almost entirely during the pre-crisis period. Together with the findings on discrimination in loan origination, we suggest that before the financial crisis, discrimination in loan origination was very limited but some pricing discrimination did occur. After 2007, discrimination in loan pricing disappeared, most likely as a result of the credit crunch.

In a nutshell, we find that using as a regressor the difference between the unobserved reasons behind banks’ willingness to provide a specific loan amount or not and the applicant’s reasons in applying for that loan amount, controls for substantial information in a simple model of the bank’s decision to grant the loan or not. Using this control, lowers the probability of discrimination of minority groups from about 7–8% to less than 1%. Thus, we posit that, even though not panacea, our approach lowers the omitted-variable problem to “more acceptable” levels, highlights that discrimination is much more limited than previously thought, and allows drawing better conclusions from the HDMA data and the entire gamut of U.S. loan applications. These are the contributions of research.

The following section discusses the theoretical considerations on mortgage-lending discrimination, the identification methods and empirical estimates of the most prominent existing literature, and the subprime crisis’ potential role in banks’ lending practices. Along these dimensions, Section 2 specifies our paper’s three research questions. Section 3 presents our data set and discusses in detail our empirical approach. Section 4 presents the empirical results, and Section 5 provides policy implications based on our findings and concludes the paper.

2 Theoretical considerations and empirical facts

Almost 50 years after the 1968 enactment of the Fair Housing Act,Footnote 2 and more than a generation after the Equal Credit Opportunity Act (ECOA) of 1974, the Home Mortgage Disclosure Act (HMDA) of 1975, and the Community Reinvestment Act (CRA)Footnote 3 of 1977, lending discrimination remains a key issue in the social, political, and research agenda. Several early studies posit that, in the past, mortgage lenders discriminated against disadvantaged groups of borrowers and that much of the discrimination was actually part of their lending policy (Munnell et al. 1996; Ladd 1998).

The antidiscrimination laws imply that the lenders should use only objective information on a loan’s expected return to decide whether to originate the loan, and they should avoid using the applicant’s membership in a protected group as part of their decision. ECOA clarifies that the term “protected group” encompasses a neighborhood’s racial composition.

Turner and Skidmore (1999) summarize the possible key forces that lead to discrimination.Footnote 4 Assuming that lenders are profit-maximizers, lending discrimination in its purest form implies the unlikely event that lenders are willing to forgo profits. In more plausible forms of lending discrimination, however, economic interest occurs as a possible explanation when lenders believe that personal characteristics, such as race or ethnicity, are reliable proxies for factors that affect credit risk and cannot be easily observed. In this case, lenders have the economic motive to deny credit to minority applicants based on an average approximation of that minority’s creditworthiness and not based on the economic characteristics of individual applications. Such discrimination is usually termed “statistical” or “profit-motivated”.

A more direct form of discrimination relates to prejudice (Quillian 2006) which occurs when lenders consider minority borrowers as inferior and do not want to interact with them. Such prejudice is usually referred to as “taste-based discrimination.” Notably, if local home owners and depositors of local banks prefer not to have minorities as neighbors, the local banks might be negatively inclined against minority applicants. Yinger (1986) and Ondrich et al. (2003) provide evidence for such behavior on the part of real estate agents. Cultural affinity is another possible reason for taste-based discrimination, which occurs when white loan officers exert less effort to help minority borrowers meet underwriting criteria compared with non-minority borrowers.

The literature on lending discrimination is very large and can be distinguished between loan origination discrimination and loan pricing discrimination.Footnote 5 We do not review this literature here because detailed reviews are already available.Footnote 6 We do briefly note, however, some of the relevant history behind the academic interest in lending discrimination (mostly racial discrimination) and the most influential findings as a means to guide our own research.

2.1 Discrimination in loan origination

In the 1990s, academic interest in mortgage lending discrimination exploded, mainly as a result of the HMDA database’s amendment in 1989. The HMDA database provides information on individual mortgage loans to assist, inter alia, in determining whether financial institutions are serving their communities’ housing needs and in identifying possible discriminatory lending practices. The 1989 amendment made extra information (on the outcome of loan applications, the location of the property, the applicant’s race and gender, etc.) available to the public and allowed more-coherent studies of discriminatory practices in loan origination.

However, the lack of data on crucial information considered during the underwriting process, such as credit scores, loan-to-value (LTV) ratios, debt-to-income (DTI) ratios, etc., still hampered empirical identification of a causal effect running from racial and gender characteristics to the loan origination decision. Munnell et al. (1996) took advantage of unique data collected by the Federal Reserve Bank of Boston in 1990 (38 variables, including risk of default, loan characteristics, personal characteristics, etc.) and amended the HMDA to circumvent the omitted-variable bias. Their findings showed that mortgage loan applicants who belong to a minority group have approximately an 8% higher probability of rejection than white borrowers. This effect is, of course, economically quite potent, despite the fact that the addition of the new variables significantly reduces the estimated level of discrimination.

Munnell et al. (1996), however, did not clear the picture on the actual level of discrimination. Specifically, a number of studies show that even the augmented list of variables used by Munnell et al. is insufficient (Horne 1997) and that racial discrimination is either lower (e.g., the separate studies by Day and Liebowitz 1998, and Faber 2013, estimate it to be about 2.8%) or non-existent (e.g., Harrison 1998). More-recent studies (most notably Hubbard et al. 2012 and Wheeler and Olson 2015) find that Munnell et al.’s estimates are more or less accurate.

The debate on the existence of discrimination in loan origination leads us to revisit an old empirical question, which we formulate as follows:

  • Question 1: Is there racial, ethnic, and/or gender discrimination in the origination of loans in the United States?

2.2 Discrimination in loan pricing

The second type of discrimination concerns loan pricing. The post-2000 literature argues that loan-pricing discrimination became increasingly important mainly because of the subprime boom that took place in the early to mid-2000s (e.g., Williams et al. 2005; Faber 2013; Ghent et al. 2014). The premise is that instead of excluding minority applicants, lending institutions opened up credit to African-Americans and Hispanics, but minorities had to pay higher loan spreads (Rugh and Massey 2010).

In 2004, the HMDA amended its database with partial loan pricing information. Specifically, lenders must report the spread (difference) between a loan’s annual percentage rate (APR) and the rate on Treasury securities of comparable maturity—but only for loans with spreads above designated thresholds. Thus, rate spreads are reported for some (relatively high-risk) mortgage loans. Consequently, the last 15 years have seen a boom in research on loan-pricing discrimination, using either the HMDA database alone (e.g., Avery et al. 2005) or the HMDA enhanced with additional information (e.g., Courchane 2007; Bocian et al. 2008; DeLoughy 2012; Ghent et al. 2014; Bayer et al. 2014). Most of this literature highlights the existence of significant loan-pricing discrimination against African American and Hispanic applicants.

A small part of the literature on loan-pricing discrimination considers the role of gender. Sen (2012) suggests that female applicants who visit the same lenders as identically situated male applicants receive approximately the same amount of high-cost loans. In a similar vein, Haughwout et al. (2009), and Cheng et al. (2011, 2014) downplay the importance of gender in mortgage lending discrimination, suggesting that any differences in mortgage rates either are attributable to the different “shopping behavior” of female applicants (they are more likely to choose lenders by recommendation instead of searching for the lowest rates) or exist only among African-American female applicants.

Based on the foregoing discussion, we formulate our second research question as follows:

  • Question 2: Is there racial, ethnic, and/or gender discrimination in the pricing of loans in the United States?

2.3 The role of the subprime crisis

Almost a decade after the global financial crisis began, a third and novel question emerges: does discrimination in mortgage lending, either in loan origination or loan pricing, differ before and after the subprime crisis? This question is based on the theoretical assumption that discriminatory policies might be exacerbated during crisis periods, which see an overall increase in economic and social uncertainty, decrease in trust, and increase in anti-social behavior.

More specifically, historical experience and academic literature shows that a financial crisis constitutes a serious challenge to protecting fundamental rights for specific groups of people. This challenge is particularly serious for the most marginalized people, who are also the most vulnerable because they probably already suffer from discrimination. For example, the International Labour Office (2011), the Agency for Fundamental Rights of the European Union (2010), and UNESCO (2009) note that a crisis feeds many types of direct or indirect discrimination because of rising economic or social inequality and insecurity. Further, the same organizations argue that during periods of turmoil, policymakers tend to de-prioritize policies targeting discrimination.

In that line, lending institutions might increasingly decide on granting mortgage loans during crisis periods by using racial and ethnical stereotypes. As in the broader social spectrum, crisis periods create severe insecurity and uncertainty in the banking sector. Given that informational asymmetries are higher during crises, banks might be less willing to take risks and be more suspicious that minority applicants lack the ability to repay a loan. Consequently, banks would reject these applicants to minimize the risk of loan default.

Correspondingly, lenders might embrace elements that reflect the role of race or ethnicity in other related markets (Reskin 2012), which affects minority borrowers’ capacity to repay loans. For instance, it has been documented that minorities face higher unemployment rates and greater decreases in wealth during economic downturns.Footnote 7 Some lenders might thus perceive that minority applicants are more likely to be unemployed or see a substantial decrease in their wealth during (or shortly after) economic downturns, and these lenders might place a higher probability of default on loan applications by minority applicants. If this perception does not follow solely from the economics of the decision to supply the loan, such lender behavior implies increased statistical discrimination during economic downturns. In fact, according to Avery et al. (2010), mortgage lending to African-Americans and Hispanics fell more quickly than the U.S. average from 2007 to 2008.

The role of the subprime crisis in discrimination, especially with respect to loan origination, has not yet been given due consideration. Based on the theoretical considerations analyzed in this subsection, we expect that, if anything, discrimination in loan origination increased after the subprime crisis erupted. The subprime crisis’ role in loan-pricing discrimination is much more nebulous. On one hand, following the same reasoning with loan origination, the subprime crisis might increase loan-pricing discrimination. On the other hand, if discrimination in loan origination increases, then loan-pricing discrimination might fall because minority applicants are not granted a loan at all.

Considering our theoretical arguments, we reiterate our third research question about the role of the subprime crisis in mortgage lending discrimination as follows:

  • Question 3: Has the level of racial, ethnic, and/or gender discrimination in loan origination or loan pricing intensified since the subprime financial crisis?

3 Data and econometric methodology

Our study aims to make progress on the identification of lending discrimination using the full gamut of mortgage loan applications across the United States, not only for specific regions or subsamples with superior data availability. In doing so, we provide a method that can serve as a policy tool for researchers, regulators, and policymakers. To fulfill this aim, we use applicant-level information solely from the HMDA for the period 2004 through 2013. Data before 2004 do not include information on ethnicity and thus limit the study on ethnicity-based discrimination. Also, the specific time period encompasses the years associated with the subprime crisis of 2007 and allows us to answer Question 3.

The HMDA provides information on a number of financial and demographic characteristics of mortgage loans. Specifically, it includes information on the requested loan amount (in thousands of dollars), loan type (conventional, insured by the Federal Housing Administration, guaranteed by Veterans’ Administration or provided by the Farm Service Agency or Rural Housing Service, etc.), loan purpose (home purchase, home improvement, or refinancing), whether or not a preapproval was requested, the owner’s occupancy status (whether or not the owner plans to occupy the home as a principal dwelling), property type (whether the application was for a one- to four-family dwelling, manufactured housing, or multifamily dwelling), loan decision details (whether the mortgage was originated, the application was denied by the financial institution or withdrawn by the applicant, etc.), lien status (whether the loan is securitized or not), and the application’s denial reason(s) if the loan was not approved.

Further, the HMDA has information on the lender’s identification number and supervisory/regulatory agency code, as well as information on the applicant’s and co-applicant’s race (American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, White), ethnicity (Hispanic/Latino or not), gender, and income (gross annual income in thousands of dollars). Finally, the property’s location information includes the identification number of the Metropolitan Statistical Area (MSA), the state and county codes, and the census tract number.

To ensure comparability across the loans, we homogenize our data set by using only applications for conventional loans for home purchases. Further, we limit the sample to securitized (by a first or a subordinate lien) loans on owner-occupied (as a principal dwelling), one- to four-family properties. Our goal is to examine whether minorities have equal housing opportunities as borrowers who belong to non-minority groups. We thus are interested only in home-purchase loans (rather than refinancing or home improvement loans), because they increase individuals’ access to homeownership. For the same reason, we focus on loans for owner-occupied as a principal dwelling properties and not for second or vacation homes.

We also remove from the analysis observations with missing data for any of the foregoing variables. The final restriction we make is to exclude applications that did not result in either originations or denials. Thus, we rule out any other type of action taken (e.g., applications withdrawn by the applicant or approved but not accepted) because the decisions made for these applications are not under the lender’s control. Following our cleansing procedure, we have a sample of 6,452,279 loan applications for the 2004–2013 period. For descriptive statistics (along with a summary of variable definitions) see Table 1.

Table 1 Variables description and summary statistics

Our identification method relies on a number of regressions on the usual “origination-denial” decision. The model’s general form is as follows:

$$ {gr}_i=f\left({a}_j,{y}_i,{\mathbf{x}}_{\boldsymbol{i}},{u}_{\boldsymbol{i}}\right), $$
(1)

where gri is a binary variable that takes the value one if the loan is originated and zero if the loan is denied by the bank; aj are the j parameters to estimate; yi is the applicant’s income; x is a vector of variables capturing the observed applicant’s characteristics; and ui is the stochastic disturbance.

The problem with directly estimating Eq. (1), using the full gamut of the loan applications in our final sample, is of course the lack of crucial underlying factors and information on borrowers’ creditworthiness, including (but not limited to) credit score variables (e.g., FICO score), LTV ratio, DTI ratio, and down payment size. These criteria are privately observed by the lending institutions and play a crucial role when a bank assesses an applicant’s riskiness and decides whether or not to originate the loan. In turn, these criteria can be severely correlated with variables such as race, ethnicity, and gender, and their exclusion from Eq. (1) leads to overestimation of the effect of minority characteristics.

In this paper, we make progress on this omitted-variable bias by proposing a new identification method in two stages. The first stage corresponds to Eqs. (2), (3), and (4). In Eq. (2), we estimate the quantity of loan amount demanded as a function of the reported applicant’s income, as follows:

$$ {L}_i^D={a}_0^D+{a}_1^D{y}_i+{a}_2^D{d}_k+{u}_i^D $$
(2)

where \( {L}_i^D \) is the dollar-denominated amount of the loan demanded by borrower i, yi is the dollar-denominated income for borrower i, dk is a dummy variable equal to 1 for borrowers belonging to the kth census tract, \( {a}_0^D \) and \( {a}_2^D \) are parameters to be estimated by OLS and \( {u}_i^D \) is the residual associated with borrower i in this equation.

The important factor for our identification approach is \( {u}_i^D \), which includes all the unobserved (to the researchers) components of an applicant’s decision to apply for a specific loan amount, besides his or her income and the fixed effects. For example, \( {\mathrm{u}}_{\mathrm{i}}^{\mathrm{D}} \) includes the LTV ratio, applicants’ perceptions about banks’ reaction to his or her FICO score, the DTI ratio relative to the loan amount requested, etc. In fact, the list of unobserved variables can be very idiosyncratic and is non-exhaustive, thus even specialized (and confidential) data sets with more detailed information are unable to fully measure these variables.

Subsequently, we regress the loan amount supplied by the bank, which equals the demanded loan amount for originated loans and zero when the loan is denied by the bank, on applicants’ income and the same set of fixed effects. This equation takes the following form:

$$ {L}_i^S={a}_0^S+{a}_1^S{y}_i+{a}_2^S{d}_k+{u}_i^S $$
(3)

where \( {L}_i^S \) is the dollar-denominated amount of the loan supplied to borrower i (which can be equal to zero in case the loan is not granted), \( {a}_0^S \) and \( {a}_2^S \) are parameters to be estimated by OLS and \( {u}_i^S \) is the residual associated with borrower i in this equation.

Equivalently to Eq. (2), the term \( {u}_i^S \) captures the unobserved (to researchers) information that lending institutions use to assess each applicant’s riskiness and decide whether or not to grant the requested loan amount. This information includes, for example, FICO scores and LTV ratios, but it is obviously not the same with the unobserved variables \( {\mathrm{u}}_{\mathrm{i}}^{\mathrm{D}} \) used by the applicants to reach a decision regarding the quantity of the demanded loan amount. Importantly, \( {u}_i^S \) also includes any discriminatory behavior of banks against applicants’ race, ethnicity, and gender.

Next, we take the difference of the residuals between Eqs. (2) and (3), as follows:

$$ {u}_i^D-{u}_i^S={u}_i^h $$
(4)

To the extent that Eqs. (1) and (2) are robustly estimated, \( {u}_i^h \) is the difference between the applicant’s model for deciding how much money to request for a mortgage (conditional on his/her income and fixed effects) and the bank’s business model conditional on the applicant’s income and the fixed effects. This difference is precisely the reasons leading the bank to deny a loan when an individual has applied. If the applicant knew that the bank would deny the loan with certainty, then he/she would not have applied. If the difference is small, then the probability of granting the loan increases. The difference still includes any discriminatory practices concerning specific loan applications, and thus the stochastic component \( {u}_i^h \) also encompasses the potential effect of discrimination. In addition, \( {u}_i^h \) encompasses the different valuation of, for instance, FICO scores, LTV ratios, and DTI ratios (among other unobserved variables) by lenders compared with borrowers.

In the second stage of our approach, we introduce \( {u}_i^h \) in the usual binary model of loan originations given by Eq. (1), rewritten here as follows:

$$ {gr}_i={c}_0+{c}_1{y}_i+{c}_2{d}_r+{c}_3{d}_g+{c}_4{d}_e+{c}_5{d}_k+{c}_6\left({u}_i^D-{u}_i^S\ \right)+{\varepsilon}_i $$
(5)

where gri is a dummy variable equal to 1 if borrower i receives a loan (that is, if \( {L}_i^S>0 \)), dr, dg, and de are dummy variables capturing race, gender and ethnicity, c0c6 are parameters to be estimated, and εi is the residual associated with borrower i in this equation.

By including dr, dg, and de in Eq. (5), we essentially subtract these observed variables from the residual \( {u}_i^h \). Then, \( {u}_i^h \) in Eq. (5) includes only the difference in the unobserved variables between the model of the applicant (conditional on his or her income) and the bank’s business model (conditional on the applicant’s income), leaving the effect of discrimination, purified from unobserved variables, to be captured by coefficients c2 to c4.

Notably, we include in Eqs. (2), (3) and (5) year fixed effects to control for any temporal variation in our dependent variable that is common to all applicants and census-tract fixed effects to control for local housing and socioeconomic characteristics that might shape the bank’s decision.Footnote 8 Given that including fixed effects is essential, and non-linear estimators such as the logit model face convergence difficulties with such a high number of fixed effects (at least in our sample), we use a linear probability ordinary least squares (OLS) model to estimate Eq. (5).Footnote 9

Our analysis so far assumes that \( {u}_i^D \) and \( {u}_i^S \) (and thus also their difference) are estimated with no statistical bias. There are two potential sources of such bias. The first concerns the reported increase in the income misreporting (or fraud) in the U.S. mortgage market from 2002 through 2006 (Financial Crimes Enforcement Network 2006). If this increase produces a serious systematic (upward) measurement error in our income variable, we should expect that our coefficient estimates in Eqs. (2), (3) and (5) are biased. We should also note, however, that given the inclusion of income in both Eqs. (2) and (3), this bias should cancel out when taking their difference in Eq. (4). Thus, the only concern is whether the bias from the inclusion of income in Eq. (5) is systematically correlated with race, gender, or ethnicity (e.g., African-American applicants systematically misreport income more than white applicants).

We do take a number of steps, however, as remedies for the potential presence of measurement error. The first and simplest is to estimate Eqs. (2) and (3) without income and examine the sensitivity of the results. If results do not change, then the measurement error is indeed cancelled out in Eq. (4), and the only remaining error concerns the potential correlation between income misreporting and the three discrimination-related variables in Eq. (5). Then, the second step is to use the errors-in-variables model of Erickson et al. (2014), in which the regression coefficients are identified using information from the higher-order cumulants of the observed variables.Footnote 10 An alternative approach, particularly useful for identifying income fraud, is to use mortgage applications only in zip codes with mortgage-loan misrepresentation values at the lowest 5% or 10% centile of the distribution of the misrepresentation measure compiled by Piskorski et al. (2015). We further discuss these approaches below.

The second source of statistical bias could result from correlation between the discrimination dummies (dr, dg, and de) and \( {u}_i^D \) and \( {u}_i^S \). However, we note after estimation that the highest observed pairwise correlation coefficient between these variables never exceeds 0.0645. Further, we show that orthogonalizing these variables produces minimal differences in the results.

The equivalent to Eq. (5) loan-pricing equation takes the form:

$$ {Spread}_i={b}_0+{b}_2{d}_r+{b}_3{d}_g+{b}_4{d}_e+{b}_4{y}_i+{u}_i^S+{\varepsilon}_i^{\prime } $$
(6)

where Spread is the difference between a loan’s APR and the rate on Treasury securities of comparable maturity. Given that spreads are available only for loans with spreads above designated thresholds, the sample is significantly smaller compared with the equivalent of discrimination in loan origination.

Identification of Eq. (6) relies only on the inclusion of \( {u}_i^S \) because Eq. (6) is estimated using only originated loans. In other words, this is a de facto supply equation and does not concern the choice of individuals to apply for the loan. Thus, the bank, in setting the interest rate on the loan, is not concerned with the unobserved characteristics related to individuals’ decisions to apply. The bank is only concerned with their own model, which is given by Eq. (3).Footnote 11 This makes using Erickson et al. (2014) method even more suitable in this case, so as to account for the measurement error in both Eqs. (3) and (6). We demonstrate, however, that changes in the results between the estimation of all models with OLS and with Erickson et al. (2014) method does not significantly change the main message of our estimates.

4 Empirical results

4.1 Discrimination in loan origination

Table 2 reports the baseline results from Eq. (5); the results from Eqs. (2) and (3) are available on request. In general, we find no significant differences whether using income in the first stage of the model (Columns I–III) or not (Columns IV–VI). Thus, we can infer that any significant measurement error related to the inclusion of income in Eqs. (2) and (3) cancels out by taking the differences in the residuals in Eq. (4).

Table 2 Discrimination in loan origination: Baseline results. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (5), given estimates from Eqs. (2)–(4). The dependent variable is Action type, and all variables are defined in Table 1. Equations (2), (3) and (5) include year and census-tract fixed effects. Columns I and IV report the results for the period 2004–2013, Columns II and V the results for the pre-crisis period (2004–2007), and Columns III and VI for the crisis and post-crisis period (2008–2013). Columns I–III report the results when income is included in the first stage of the model, and Columns IV–VI when income is not included in the first-stage regressions

The results for the full period (Column I) show that race, gender, and ethnicity bear a statistically significant coefficient at the 1% level. The economic significance of these effects, however, is quite lower than expected. Specifically, we find that white applicants have 0.39% higher probability to be granted a home mortgage loan than African-Americans, while non-Hispanic applicants have a 0.25% higher probability than Hispanic applicants. For gender, the coefficient shows that male applicants have a 0.06% higher probability to be denied a loan relative to female applicants, which economically is very small. As expected, the coefficient estimate on the first-stage residual differential, \( {u}_i^h \), is highly statistically significant.

Evidently, our estimates are significantly lower than equivalent estimates from existing studies on mortgage lending discrimination. For example, Munnell et al. (1996) and Hubbard et al. (2012) find an approximately 8% higher probability that white applicants will be granted a loan compared with African-American applicants, and Wheeler and Olson (2015) find an equivalent 9.8%. Notably, if we estimate Eq. (5) without the stochastic component \( {u}_i^h \), the coefficient on race becomes 0.128, on ethnicity −0.085, and on gender 0.002 (results in Table 12 in the Appendix). These estimates are very close to the estimates of past research that solely use the HMDA database to infer lending discrimination.Footnote 12 Also, the adjusted R-squared decreases from 96.6% in the model with \( {u}_i^h \) to 8.4% in the model without \( {u}_i^h \). Given the foregoing, we must attribute the very large drop in the discrimination-related probabilities to the very significant reduction in omitted-variable bias achieved by including \( {u}_i^h \) in Eq. (5). In this respect, our approach contributes to the literature on lending discrimination and allows for more-accurate estimation of discriminatory practices using the full gamut of loan applications in the HMDA database across the United States.Footnote 13

In the regressions of Columns II and III (or equivalently, in those of V and VI for the models without income in the first stage), we split our sample into the pre- and post-2007 periods. We find that during the pre-crisis period, white applicants have a 0.23% higher probability to be granted a mortgage loan than African-American applicants (Column II). As hypothesized in Section 2, this probability increases after the onset of the subprime crisis in 2007. In particular, during the post-2007 period (2008–2013), we find a nearly three times higher probability (0.66%) that white applicants are granted a loan compared with African-American applicants ceteris paribus (Column III).

The role of the applicant’s ethnicity seems also to be significantly more important after the subprime crisis. Specifically, during the post-2007 period, Hispanic applicants have a 0.54% lower probability of receiving a loan than non-Hispanics (almost four times higher than the 0.14% of the pre-crisis period). The estimated coefficient for the gender variable is quite low in the pre-crisis period, when female applicants had a slightly higher probability (0.07%) to receive a loan than male applicants. This probability diminishes during the post-2007 period (0.01%), and it is statistically insignificant at any conventional level.

We also experiment with estimating Eq. (5) by year, and Fig. 1 reports the coefficient estimates in graphical form. Apparently, the annual estimates of race, gender, and ethnicity are in line with the preceding suggestion that loan origination discrimination levels are particularly low before the subprime crisis and that a slight increase occurred after 2007 in the absolute values of the estimates of race and ethnicity. Still, we cannot suggest that extensive discrimination takes place against African-American or Hispanic applicants because the maximum (absolute) value on both race and ethnicity is about 0.6%.

Fig. 1
figure 1

Discrimination in loan origination by year

The regressions of Table 3 replicate those of Table 2 using Erickson et al.’s (2014) errors-in-variables model for estimating Eq. (5). Following the proposed procedure in Erickson et al., we first use the within transformation to account for the census tract and year fixed effects and then use the transformed variables in Eq. (5), assuming that both the transformed \( {u}_i^h \) and y variables are measured with error.

Table 3 Discrimination in loan origination: Error-in-variables model. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (5), given estimates from Eqs. (2)–(4). The dependent variable is Action type, and all variables are defined in Table 1. Equations (2), (3) and (5) include year and census-tract fixed effects. Equation (5) is estimated using the Erickson–Whited (2002) errors-in-variables model. Columns I and IV report the results for the period 2004–2013, Columns II and V the results for the pre-crisis period (2004–2007), and Columns III and VI for the crisis and post-crisis period (2008–2013). Columns I–III report the results when income is included in the first stage of the model and Columns IV–VI when income is not included in the first-stage regressions

The results show that, if anything, the coefficient estimates on race, ethnicity, and gender are a bit smaller than the equivalent in Table 2. Thus, the role of measurement error seems to be minimal and, if anything, producing an upward bias in the results. We also find the same pattern concerning the role of the crisis, with the estimates in the post-2007 period being somewhat larger than those of the pre-crisis period. Even in the post-2007 period, however, the estimates’ economic significance is fairly low, indicating for example that white applicants are 0.49% more likely to have a loan approved compared with African-American applicants, ceteris paribus.

In Table 4, we include bank fixed effects in Eq. (5).Footnote 14 Thus, these specifications control for each bank’s business model and consider within-bank discrimination of mortgage loan applicants. The results are almost the same as those of Table 2.

Table 4 Discrimination in loan origination: Including bank fixed effects. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (5), given estimates from Eqs. (2)–(4). The dependent variable is Action type, and all variables are defined in Table 1. Equations (2), (3) and (5) include year, bank, and census-tract fixed effects. Column I reports the results for the period 2004–2013, Column II the results for the pre-crisis period (2004–2007), and Column III for the crisis and post-crisis period (2008–2013). In all regressions, income is included in the first stage of the model

Further, in Table 5, we examine whether our findings differ by supervisory agency. We split our sample into three groups based on the supervisor being the OCC, the FRS, or the FDIC. We find some evidence that, at least in the full sample period, discriminatory practices are lowest in the OCC-supervised banks and highest in the FDIC-supervised banks. This result is somewhat expected because in general, the FDIC supervises the relatively small local banks. Nonetheless, the differences and magnitude of the coefficients show a very low level of discriminatory practices irrespective of the supervisor.

Table 5 Discrimination in loan origination: Evidence by the three main supervisory authorities. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (5), given estimates from Eqs. (2)–(4). The dependent variable is Action type, and all variables are defined in Table 1. Eqautions (2), (3) and (5) include year and census-tract fixed effects. Columns I, IV, and VII report results for the period 2004–2013, Columns II, V, and VIII for the pre-crisis period (2004–2007), and Columns III, VI, and IX for the crisis and post-crisis period (2008–2013). Columns I–III report results for banks supervised by the OCC, Columns IV-VI for the banks supervised by the Fed, and Columns VII-IX for the banks supervised by the FDIC

So far, we have assumed that nonlinear effects between the different forms of discrimination in loan origination are not present. To relax this assumption, we introduce in Eq. (5) a number of interaction terms between (i) race and gender, (ii) race and ethnicity, and (iii) gender and ethnicity. We further refine our results by including the triple interaction between race, gander, and ethnicity. In Table 6 we report our findings, with the majority of the interaction variables being statistically insignificant. Specifically, the coefficient estimates on the interaction term between race and ethnicity and the triple interaction term among race, gender, and ethnicity are statistically insignificant, while the interaction term between race and gender is positive and significant only for the post-crisis period. The interaction term between the gender and the ethnicity is positive and significant at the 10% level indicating that the probability of male applicants to be granted a loan increases when the applicants belong to a Hispanic minority group. Even in this case, however, the effect is economically very small.

Table 6 Discrimination in loan origination: Interaction terms between the discrimination variables. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (5), given estimates from Eqs. (2)–(4). The dependent variable is Action type, and all variables are defined in Table 1. Eqautions (2), (3) and (5) include year, bank, and census-tract fixed effects. Column I reports the results for the period 2004–2013, Column II the results for the pre-crisis period (2004–2007), and Column III for the crisis and post-crisis period (2008–2013). In all regressions, income is included in the first stage of the model

An important sensitivity test is to use mortgage applications only in zip codes with mortgage-loan misrepresentation values at the lowest 5% centile of the distribution of the misrepresentation measure compiled by Piskorski et al. (2015). Specifically, we adopt the following procedure: (1) We match the census tracts in the HMDA annual files with zip codes using the Missouri Census Data bridge for the 2000 census.Footnote 15 For a one-to-one match between census tracts and zip codes, we rely on population weights. We use only zip codes for which census tracts cover more than 85% of a zip code’s population. (2) We match the zip codes with the misrepresentation measure compiled by Piskorski et al. (2015). Although these data refer to the 2005–2007 period and concern misrepresentations with regard to owner-occupied property or deliberately omitted information on a second lien, Mian and Sufi (2015) show that this type of misrepresentation is also strongly related to fraudulent income overstatement. Moreover, Mian and Sufi (2015) show that mortgage fraud is a fixed characteristic of such zip codes (does not have a significant time variation) and thus could be used for the full sample period. (3) For our sensitivity analysis, we keep only those applications in HMDA annual files for which the misrepresentation measure is at the low 10% centile of its distribution. The results, reported in Table 14 in the Appendix, show that like the error-in-variables model, income misreporting does not affect our results.

A final sensitivity test involves orthogonalizing the residuals in Eqs. (2) or (3) relative to dr, dg and de before they are plugged into Eq. (5). Even though the pairwise correlation coefficients never exceed 0.0645, this exercise aims to further reduce potential endogeneity stemming from such correlation. The results, reported in Table 15 of the Appendix, are very close to our baseline results in Table 2.

4.2 Discrimination in loan pricing

In Table 7, we report our baseline results from estimating Eq. (6). For expositional brevity, we report only the results from specifications that include income in the first stage of the model (again, this decision does not affect the results). Column I reports the results for the full sample and shows that white applicants pay a lower spread on the originated loans by 0.37 basis points.Footnote 16 This finding holds despite the inclusion of \( {u}_i^S \) in Eq. (6) and its very high statistical and economic significance.

Table 7 Discrimination in loan pricing: Baseline results. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (6), given estimates from Eq. (3). The dependent variable is Spread, and all variables are defined in Table 1. Equations (3) and (6) include year and census-tract fixed effects. Column I reports the results for the period 2004–2013, Column II the results for the pre-crisis period (2004–2007), and Column III for the crisis and post-crisis period (2008–2013). In all regressions, income is included in the first stage of the model

Interestingly, this difference comes from the pre-crisis period. Column II of Table 7 shows that during the pre-crisis period, the average white applicant paid a lower spread on the originated loans by 0.39 basis points, whereas in the post-2007 period, the relevant coefficient is statistically insignificant. This finding is intuitive conditional on the results discussed in Section 4.1. In the pre-crisis period, deemed a low-risk period, banks were willing to originate the loans but required somewhat higher spreads from minority applicants. In the post-2007 period, deemed a very risky period, the probability is slightly higher that banks will discriminate by not granting the loan.

Concerning the effect of ethnicity, the results are economically weaker and show that Hispanic applicants obtain mortgage loans with approximately 0.11 basis points higher spread. Similar to race, the effect is considerably higher in the pre-crisis period. Finally, the effect of gender is even less potent, with male applicants paying 0.05 lower spreads in the whole sample. When we differentiate between the pre-2007 and post-2007 periods, we find that in the risky post-2007 period, women are given lower spreads by approximately 0.06 basis points. Even though this effect is economically small, we can attribute such bank behavior to the general perception that women are more risk averse in their financial decisions and that banks price in this risk aversion during periods of high uncertainty.

In Table 8 we replicate the regressions of Table 7 using the errors-in-variables model. We work under the assumption that both the residual of Eq. (3), \( {u}_i^h \), and applicants’ income are measured with error. Using this model is potentially more important in the loan pricing equation than in the loan origination equation, because the former does not eliminate bias through Eq. (4). Still, changes in the results are not significant, implying low measurement error in \( {u}_i^h \) and income.

Table 8 Discrimination in loan pricing: Error-in-variables model. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (6), given estimates from Eq. (3). The dependent variable is Spread, and all variables are defined in Table 1. Equations (3) and (6) include year and census-tract fixed effects. Equation (6) is estimated using the Erickson–Whited (2002) errors-in-variables model. Column I reports the results for the period 2004–2013, Column II the results for the pre-crisis period (2004–2007), and Column III for the crisis and post-crisis period (2008–2013). In all regressions, income is included in the first stage of the model

In Table 9, we consider including bank fixed effects in Eq. (6). Interestingly, the coefficient estimates on race, gender, and ethnicity decrease somewhat. For example, white applicants obtain loans with a lower spread by 0.27 basis points, whereas the equivalent spread in the model without bank fixed effects is 0.37 basis points. This finding is expected because in the Table 9 regressions we consider only within-bank discrimination, whereas in previous tables we also consider across-bank discrimination. The pattern in the results regarding the effect of the subprime crisis is very similar to the one in Table 7.

Table 9 Discrimination in loan pricing: Including bank fixed effects. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (6), given estimates from Eq. (3). The dependent variable is Spread, and all variables are defined in Table 1. Equations (3) and (6) include year, bank, and census-tract fixed effects. Column I reports the results for the period 2004–2013, Column II the results for the pre-crisis period (2004–2007), and Column III for the crisis and post-crisis period (2008–2013). In all regressions, income is included in the first stage of the model

We also repeat our analysis on the three main regulatory agencies. The findings, reported in Table 10 indicate that our baseline results from Eq. (6) are identical with the ones obtained from banks supervised by the FRS. This finding is expected given that approximately 65% of the higher-priced mortgage loans in our sample are originated by the above banking institutions. Interestingly, discriminatory behavior is the lowest in banks that are under the jurisdiction of the FDIC. The average white applicant in these institutions pays approximately a 0.18 basis points lower spread on the originated loans which is a substantially lower difference compared with the 0.34 and 0.25 basis points lower spread that white individuals pay in FRS-supervised and OCC-supervised banks, respectively.

Table 10 Discrimination in loan pricing: Evidence by the three main supervisory authorities. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (6), given estimates from Eq. (3). The dependent variable is Spread, and all variables are defined in Table 1. Equations (3) and (6) include year and census-tract fixed effects. Columns I, IV, and VII report results for the period 2004–2013, Columns II, V, and VIII for the pre-crisis period (2004–2007), and Columns III, VI, and IX for the crisis and post-crisis period (2008–2013). Columns I–III report results for banks supervised by the OCC, Columns IV–VI for banks supervised by the Fed, and Columns VII–IX for banks supervised by the FDIC

Symmetrically to the relevant tests for Eq. (5), we introduce in Eq. (6) interaction terms to examine possible nonlinear effects in the three forms of discrimination. We report our findings in Table 11. We note that there are some nonlinear effects originating in the pre-crisis period. Specifically, the 0.36 points lower spread paid by the average white applicant is by 0.25 points higher for white Hispanic applicants in the period 2004–2007. Further, the interaction term between gender and ethnicity shows that white male applicants pay an extra 0.07 points lower spread compared to white female applicants in the period 2004–2007. These nonlinear effects, even though economically small, are in line with the results of some discrimination in loan pricing, especially in the pre-crisis period. The coefficient on the triple interaction term between race, gender, and ethnicity is statistically insignificant.

Table 11 Discrimination in loan pricing: Interaction terms between the discrimination variables. The table reports coefficient estimates and t-statistics (in parentheses) from the estimation of Eq. (6), given estimates from Eq. (3). The dependent variable is Spread, and all variables are defined in Table 1. Equations (3) and (6) include year and census-tract fixed effects. Column I reports the results for the period 2004–2013, Column II the results for the pre-crisis period (2004–2007), and Column III for the crisis and post-crisis period (2008–2013). In all regressions, income is included in the first stage of the model

Finally, we check whether our results change when using a truncated regression and maximum likelihood. The estimator is preferred when the depended variable is censored, as is in the case with Spread, which has a lower limit equal to 1.5 basis points. We report the relevant results in Table 16 of the Appendix. The results are essentially the same with those of Table 7; again, we attribute this similarity to our very large data set with dense observations.

5 Conclusions and policy implications

This paper revisits the issue of mortgage lending discrimination in the United States and proposes a method that lowers the omitted-variable bias stemming from the use of the HMDA database. We study discrimination based on race, ethnicity, and gender, using loan applications from 2004 through 2013, which includes the subprime crisis. Further, we identify both discrimination in loan origination (i.e., the bank’s decision to grant or deny the loan for reasons related to race, ethnicity, and gender) and in loan pricing (i.e., the bank’s decision to grant different loan spreads for reasons related to race, ethnicity, and gender).

Our results show that the effect of race and ethnicity in a bank’s decision to grant a mortgage loan is significantly lower compared with the corresponding results from previous studies on mortgage lending discrimination. Specifically, according to our preferred specification, white applicants have only a 0.39% higher probability to be granted a mortgage loan than African Americans, while non-Hispanic applicants have a 0.25% higher probability than Hispanic applicants. For gender, the coefficient estimates are statistically insignificant. In contrast, when we use the main methodology of previous studies, we find estimates close to or higher than these studies (about 12% for race and 8.5% for ethnicity). We thus believe that our empirical strategy, even though not panacea, captures a great deal of the omitted-variable bias and allows for more-accurate estimation of discriminatory practices across the whole spectrum of insured U.S. banking institutions.

Our analysis does not suggest by any means that lending institutions do not engage in discriminatory practices. We simply posit that the average estimates that 8 or more out of 100 applicants are denied mortgage loans based on their race are inaccurate. In contrast, our methodology provides an instrument for (1) academics to identify discriminatory practices using basic data across the entire United States, (2) regulators to identify discrimination beyond doubt and punish it with enforcement actions, and (3) policymakers to initiate new laws and regulations that prevent discrimination.

With respect to loan spreads, we find that white and non-Hispanic applicants are charged 0.37 and 0.11 basis points lower spreads on the originated loans respectively, than African-American and Hispanic applicants. We also show that these results come almost entirely from the pre-crisis period. Combined with our finding that racial discrimination in loan origination increases from 0.23% in the pre-crisis period to 0.66% during the crisis and post-2007 period, our finding on discrimination in loan pricing suggests that after the crisis erupted, discrimination simply shifted from loan pricing to loan (non)-origination. This finding itself provides evidence that discriminatory practices, even though relatively isolated events, do take place and require caution. Further, our finding reminds us of the old and influential economics and sociology literatures suggesting that crises mostly hurt minorities and steps are needed to protect these minorities during economic downturns.