1 Introduction

As the largest single buyer of goods and services in the world, the federal government of the USA has long sought to affect marketplace outcomes beyond the immediate objectives of procurement. The small business administration (SBA) is empowered by congress to oversee a wide array of preferential contracting programs. Legislation directs percentages of federal contracts be awarded to businesses owned by various social and economic categories of citizen. In addition to broad agency percentage goals, contracting officers are empowered to set aside contracts for competition only among firms owned by members of preferred socioeconomic groups and in some cases to award such contracts without competitive bidding. Subsidized loans and subcontracting incentives also benefit these select categories of business owners. Preferred groups include small disadvantaged businesses (which benefit from a legislated goal of 5% of total federal spending) and service-disabled veteran-owned small businesses (3%) (Small Business Act Sec. 15(g)(1)). The 8(a) business development program, named for Section 8(a) of the Small Business Act and administered by the SBA, goes beyond sole-source contracts and similar direct monetary benefits to provide broad support to select small disadvantaged businesses (which are usually minority-owned) through management and technical assistance, mentoring programs, and facilitated alignment with government requirement owners. Table 1 summarizes the benefits of these programs. This analysis examines the effectiveness of the 8(a) program in achieving its stated goals of fostering growth among disadvantaged businesses and encouraging them to graduate from the program as viable business concerns able to compete in the broader marketplace.

Table 1 Small business certifications
Full size table

Prior evidence examining the effectiveness of similar government contracting preference programs has produced mixed results. Some studies find that well-executed programs may indeed be successful at affecting market outcomes (Strupler and Wolter 2016; Boston and Boston 2007); others suggest that they are likely to be ineffectual (Black 1983), encouraging gaming of the system (Bates and Williams 1995) or encouraging minorities to invest their energies in non-viable business plans (Bates 2004). More recent evidence from Acs et al. (2016) supports the claim that programs aimed at encouraging entrepreneurship in target groups mostly waste taxpayer money, funding those already motivated to start businesses and generating single proprietorships with no motivation to innovate. Astebro (2017) goes further, presenting evidence that the earnings gap between wage-earning employees and entrepreneurs in developed countries is illusory and there is no clear reason to incentivize such behavior with subsidies.

An underlying theme of these studies is that effectiveness depends largely on program characteristics and local circumstances. Butler et al. (2016) note that a small-scale program in Argentina was effective at stimulating growth in entrepreneurial firms, while Figueroa-Armijos and Johnson (2016) examine a similar program in Kansas and find no effects. By a wide margin, the U.S. federal government operates the largest preferential contracting program in the world designed to stimulate entrepreneurship in targeted socioeconomic groups, and yet its program has received remarkably little direct scrutiny. Most studies of it have been anecdotal, examining individual cases of fraud and mismanagement. This paper addresses this gap with the first large-scale empirical evaluation of the 8(a) program, building on previous examinations of diverse state and local preferential contracting programs.

Theory does not offer clear answers on whether contracting preference programs based on socioeconomic characteristics of business owners will be effective. It is possible that historical patterns of discrimination have left “money on the table” in the form of more productive firms that fail in the competitive market for reasons orthogonal to ability (e.g., skin color). Alternatively, selecting federal contractors based on characteristics unrelated to contract performance may represent a pure subsidy to lower-performing firms and an impetus to costly rent-seeking and political competition. Higher costs to the government may or may not produce the intended results (Stigler 1971, Becker 1983). I examine these contrasting theories to determine which best fits the data.

The empirical design employed differentiates short-run measures of success driven directly by preferential contract terms from longer-run success driven indirectly by business acumen fostered in the target population of disadvantaged business owners. The investigation takes advantage of differences between U.S. federal small business programs targeted at different socioeconomic categories to distinguish the contribution to firm performance of administrative support and other management assistance, distinct from the direct effects of subsidies. It compares 8(a) participants to service-disabled, veteran-owned small businesses which enjoy similar contract set-asides and sole-source awards, but not the broader range of non-monetary benefits available to 8(a) firms. It controls for personal characteristics of business owners such as social disadvantage and veteran status.

I link procurement data from the Federal Procurement Data System (FPDS) to employment and revenue figures from the National Establishment Time Series (NETS) using firm DUNS numbers reported in both systems. The result is a detailed dataset of over 16,000 small businesses which received contract awards between 2007 and 2012. Contract awards in the data set amounted to $28.8 billion in 2012, representing approximately 32% of total federal spending to small businesses. Of these firms, 6713 are small disadvantaged businesses, 1621 are service-disabled veteran-owned small businesses, and 3738 are participants in the 8(a) business development program.

Cursory examination of the data suggests that the 8(a) program has strong positive effects on firm performance. Simple comparisons with non-participants, however, fail to account for the direct effects of subsidies. They do not assess the broader ability of the program to produce viable businesses which can survive without preferential treatment. Participants in the 8(a) program receive large amounts of funding relative to non-participating peers, making it challenging to disentangle immediate effects of contract awards from the longer-run impact on firm viability. I account for these problems by comparing program participants with similar government contractors that receive comparable contract subsidies without the broader array of support provided to 8(a) firms. Here, a more complicated picture emerges. The average growth of 8(a) firms over 6 years is nearly identical to that of service-disabled veteran-owned small businesses, which receive comparable contract subsidies but no management and technical assistance. More importantly, 8(a) firms are no more likely than baseline non-preferred firms to remain in business; they perform substantially worse than service-disabled veteran-owned firms by this measure.

Section 2 reviews implementation details of socioeconomic programs, previous research into their efficacy and the economic theory behind them. Section 3 describes the dataset and empirical approach. Section 4 presents results and tests of robustness. Section 5 discusses the significance of these findings and concludes.

2 Background and theory

Efforts to achieve policy goals through federal contracting began in earnest with the Small Business Act of 1953, although similar programs existed in the 1940’s as part of the New Deal and the war effort. The justification for activist contracting policy changed over time, as described by Anglund (2000). The earliest arguments emphasized fairness, and not until the 1960’s and 1970’s did the focus shift to civil rights (Kotlowski 1998). In the 1980’s and 1990’s, the focus shifted again, this time to economic impact. With economists highlighting the dynamism of small businesses, legislators began to use job creation, innovation and exports as justifications for preference programs. Subsequent evidence caveating the economic impact of small businesses did little to curb legislative enthusiasm for subsidies (Davis et al. 1996; Decker et al. 2014).

Amounts involved are not insignificant. According to the most recent scorecard published by the SBA, in fiscal year 2015, agencies awarded over $90 billion to small businesses, including $35 billion to small disadvantaged businesses (see Fig. 1) and $14 billion to service-disabled veteran-owned small businesses (Fig. 2). Nevertheless, remarkably little research has examined the effectiveness of these programs in stimulating growth of businesses in targeted socioeconomic groups.

Fig. 1
figure 1

Contract spending to small disadvantaged businesses. The federal government’s goal for contract awards to small disadvantaged businesses (SDB) is 5% of total spend. The SDB category includes 8(a) firms. Data are published at https://www.sba.gov/contracting/finding-government-customers/see-agency-small-business-scorecards

Full size image
Fig. 2
figure 2

Contract spending to service-disabled veteran-owned small businesses. The federal government’s goal for contract awards to service-disabled veteran-owned small businesses is 3% of total spend. Data are published at https://www.sba.gov/contracting/finding-government-customers/see-agency-small-business-scorecards

Full size image

2.1 Regulatory context

The U.S. government uses various tools to direct contracts to favored groups (Anglund 2000, Kotlowski 1998, Leiter and Leiter 2002, McVay 2009). The most basic of these are the broad agency goals outlined above, through which predetermined percentages of contract spending are directed to preferred categories of business owners. The SBA assists in coordinating the efforts of federal agencies to ensure the government as a whole meets socioeconomic procurement targets. Although there are no explicit penalties for failing to meet these goals (and the government has only begun consistently to do so in recent years), pressure is strong at the agency level to comply. Regulations offer tools to assist contracting offices in meeting their goals. Contract set-asides, the most common tool, reserve a particular award for competition only among firms in a given socioeconomic category. Members of some favored categories of firms, including 8(a) participants and service-disabled veteran-owned small businesses, may also receive sole-source contracts without a requirement for competition. Table 2 summarizes the benefits available to the socioeconomic categories examined here.

Table 2 Available benefits by program
Full size table

The 8(a) business development program provides the most comprehensive set of benefits to disadvantaged businesses. Participants must be small businesses at least 51% owned and controlled by socially and economically disadvantaged U.S. citizens. Social disadvantage is defined by law as those subject to racial or ethnic prejudice or cultural bias due to identification as a member of certain groups. An applicant must demonstrate personal experiences of substantial disadvantage within the USA and consequent negative impact on participation in business. Proof of economic disadvantage is also required and includes a narrative description of personal circumstances as well as submission of financial records. Personal assets may not exceed $4 million, average annual personal income may not exceed $250,000 over the previous 3 years, and adjusted net worth may not exceed $250,000 (these values are occasionally adjusted and were lower at the time the data here were collected).

Upon application and admittance to the 8(a) program, participants progress through two phases. The first is a 4-year developmental stage in which firms are eligible for additional benefits such as the Mentor-Protégé Program. The second is a transition phase in which firms prepare to exit the program and survive without its associated benefits. Throughout this 9-year period, participants are eligible for sole-source contracts of up to $4 million for goods and services and $6.5 million for manufacturing, up to a program cap of $100 million. The SBA coordinates with other government agencies to facilitate these awards and encourage contracting officers to consider 8(a) participants when filling requirements. Agencies may in effect pass requirements to the SBA, which then allocates funding on a set-aside or sole-source basis to 8(a) firms. Participating firms may form joint ventures to take on larger contracts, and through the Mentor-Protégé Program may partner with other established government contractors which assist in contract performance. Protégé firms may form joint ventures with mentor firms or relinquish to them up to 40% of ownership in return for capital investment.

The SBA monitors participating firms throughout the program to ensure they maintain a balance between commercial and government business. District offices work with firms within their regions by conducting annual reviews and monitoring participants’ business planning activities. Administrators provide business training, counseling, marketing assistance, and executive development through government resources or contracted partner firms. Participants may gain access to surplus government property, SBA-guaranteed loans, and bonding assistance.

The Small Business Act enumerates testable hypotheses regarding the intended effects of the 8(a) business development program. Apart from normative issues of social justice not considered here, it specifies the following:

  1. 1.

    “[T]he conditions of socially and economically disadvantaged groups…can be improved by providing the maximum practicable opportunity for the development of small business concerns…”

  2. 2.

    “[S]uch development can be materially advanced through the procurement by the United States of articles, equipment, services, materials, and construction work from such concerns…”

  3. 3.

    “[S]uch procurements also benefit the United States by encouraging the expansion of suppliers for such procurements, thereby encouraging competition among such suppliers and promoting economy in such procurements….”

(Small Business Act Sec. 2(f)(1)).

Thus the explicit goals of the program are to (1) provide opportunities in business for disadvantaged groups, (2) encourage expansion of participating firms, (3) encourage competition among them, and (4) economize on government spending through purchasing in a more competitive environment.

2.2 Prior evidence

Substantial analysis supports the claim that historically disadvantaged groups experience poorer outcomes in the business environment. Firms owned by them tend to by younger, more highly leveraged, and less profitable (Bates 1985). Fairlie and Robb (2007, 2008) propose that the problem is one of historical opportunities. Minorities are less likely to have self-employed family members and are thus less likely to have entrepreneurship exposure that would contribute to success. The lingering effects of discrimination are perpetuated through low family- and peer-group educational opportunities. Cultural differences are also critically important (Minola et al. 2016). Service-disabled veterans and women face challenges of their own, experiencing lower success rates in business relative to comparable peers from other socioeconomic groups (Cox and Moore 2013; Rosa et al. 1996).

Minorities are also harmed through discrimination in credit markets; many studies find that minorities face strong headwinds in financing business activities (Blanchard et al. 2008; Blanchflower et al. 2003; Cavalluzzo and Cavalluzzo 1998; Cavalluzzo and Wolken 2005). Other analyses examine the legal challenges to awarding contracts based on socioeconomic characteristics (Hopkins 1975, Rice 1992, Reeder and Vergilio 1984, Sirmons 2004, Sakallaris 2007).

Governments around the world have tried to overcome such biases (McCrudden 2004), and many studies evaluate the efficacy of affirmative action (Ashenfelter and Heckman 1976, Holzer and Neumark 2000, Marion 2009, Myers and Yuan 2013). The method of overcoming historical bias examined here is the application of public contracting policy to markets for goods and services in which governments play a major role. Some researchers who have examined these types of programs at the state and local levels found non-existent or counterproductive effects (Bates 2009; Davila et al. 2012; Enchautegui et al. 1996; Myers and Chan 1996; Sweet 2006). One of the earliest examinations looked at over 4000 minority business enterprises that sold goods and services to state and local governments in 1987 (Bates and Williams 1996). Following up on them 4 years later, the authors found that those with substantial portions of revenue dependent on government contracting were more likely to go out of business. The authors hypothesize that many of these businesses may have been front companies for larger business concerns which disappeared after the contract in question concluded. La Noue (2008) examines an agency-specific race-conscious contracting program at the U.S. federal level and finds similar evidence of poor implementation.

Other studies find positive effects. Chatterji et al. (2014) look at city level contract set-aside programs and find that the gap in business ownership rates between blacks and whites fell by 3 percentage points following implementation, although there was little discernible effect on minority employment overall. Bates and Williams (1995) suggest that success of preferential contracting programs depends on their design; poorly designed policies encourage the creation of front companies that pass on contracts to established firms. Bates and Williams’ finding that well-designed programs can be successful is corroborated by more recent evidence (Bates 2015).

Government support to disadvantaged groups of citizens depends upon the broader ability of public policy to stimulate entrepreneurship. Evidence that they can effectively do so is similarly mixed. Supporting the hypothesis of Bates and Williams (1995), recent studies of small-scale programs find that some are successful in fostering entrepreneurship while others are not (Butler et al. 2016; Figueroa-Armijos and Johnson 2016). A recent survey of the evidence concludes that programs directly designed to encourage start-ups have perverse effects and that policy should rather address the problem of falling entrepreneurship indirectly by, for example, reducing healthcare-related distortions and improving STEM education (Acs et al. 2016).

Even though U.S. government contracting programs designed to address socioeconomic outcome disparities are the largest in the world by a wide margin, there are few large-scale empirical studies testing their effects. Using early data, Black (1983) finds that these federal programs do not substantially increase the amount of funding going to minority-owned businesses. Some qualitative assessments are critical. Bates (2004) claims that the programs are “flawed in intent, design and implementation;” they assume that capital acquired through debt and proprietor human capital are substitutes rather than complements. Minorities may receive little substantive assistance, or may be encouraged to proceed with non-viable business models. However, such assessments remain speculation without broad empirical support. The analysis presented here addresses this gap. It examines whether the billions of dollars a year directed by the U.S. government toward encouragement of entrepreneurship among disadvantaged groups is well-spent. It provides the most substantial empirical evidence to date that explicit encouragement of entrepreneurial behavior is fraught with unintended consequences.

3 Dataset and empirical approach

The Federal Procurement Data System maintains records of all significant government contracts and tracks a wealth of contract attributes including the socioeconomic status of awardees. Contracting officers throughout the federal government enter data into the system, which are then aggregated and used to monitor various performance metrics such as small business utilization rates. Data in the system are prone to error, but improved significantly upon passage of the Federal Funding Accountability and Transparency Act of 2006. In particular, awardees from 2007 onward are required to obtain DUNS numbers which are then used to track firms in FPDS. For this reason, the analysis here includes only contracts awarded after 2006. Dun and Bradstreet, the originator of the DUNS number system, also uses the numbers in monitoring firms’ credit quality. Credit tracking information includes annual reporting of firm employment and sales receipts, and is published in the NETS database. By linking FPDS data to NETS data by firm DUNS number, I pair federal contracts with awardee characteristics.

The matching procedure also assists in cleaning error-prone FPDS data; of more than 120,000 small business establishments active in FPDS over the period examined, I match 21,089 with DUNS numbers extracted from the NETS. To simplify the analysis, a further 505 firms consisting of more than one establishment were removed. Eliminating firms with zero employment or sales throughout the period brings the dataset to 19,855 observations. I remove firms annotated as 8(a) participants which are not also designated as small disadvantaged businesses since SDB certification is a prerequisite for the 8(a) program and the discrepancy implies erroneous data entry. I similarly remove firms with more than 500 employees in 2007 and those with several different headquarters DUNS numbers over the period examined (implying M&A activity). Examining only those firms with positive employment and sales in the first year of the review period produces a final dataset with 16,796 observations. Growth models examine only those firms that remained in business as of 2013; there are 14,978 of these.

3.1 Dataset description

The linked databases provide all the variables necessary for the analysis. They identify firms as small or large (by the government’s definition, which varies across different industries), disadvantaged, veteran-owned, and/or as 8(a) participants. They record firms’ location, age, credit score and industry by 8-digit Standard Industrial Classification (SIC) and 6-digit North American Industrial Classification System (NAICS) code. Most importantly, they record annual employment and sales estimates, allowing evaluation of growth trends. NETS data are current as of the first day of the recorded year and are presently available through 2013. The beginning of the dataset used here thus includes firm-level employment and sales as of January 2007 and federal awards to those firms in the following year (2007). The end of the dataset includes awards in 2012 and employment and sales figures as of the end of that year, or January 2013.

Many firms hold more than one socioeconomic classification. Small disadvantaged businesses, for example, may also be veteran-owned. Participants in the 8(a) program must be small disadvantaged businesses, of which 93% are minority-owned. Table 3 illustrates the overlap of various socioeconomic classifications by number of firms. As Table 3 shows, all 3738 8(a) firms in the sample are small disadvantaged businesses. There are 411 firms that are 8(a) participants owned by service-disabled veterans. There are 365 service-disabled veteran-owned firms that are small disadvantaged businesses but do not participate in the 8(a) program. This leaves 845 firms falling solely in the category of service-disabled veterans and 3005 firms that participate in the 8(a) program alone. There are 9238 small business in the sample that do not belong to any preferred group, 2097 of which are owned by veterans (veterans do not receive preferential treatment unless they are also service-disabled).

Table 3 SDB, 8(a), and SDVOSB program overlap
Full size table

Network effects are important to innovation and growth. Early theoretical descriptions of the phenomenon are found in Marshall (1920) and Krugman (1991), while later authors examine the effect empirically in the context of government support to R&D (Jaffe et al. 1993; Audretsch and Feldman 1996). As Fig. 3 illustrates, federal small business funding is strongly concentrated in certain regions of the USA, particularly near large cities and military installations. The models below assessing success rates of small-business contractors include geographic network effects by controlling for aggregate industry presence within the local area of a given firm.

Fig. 3
figure 3

Concentration of sampled firms by county. Counties are color-coded by total number of sampled small businesses as of 2007. Darker areas have a higher concentration of firms

Full size image

Socioeconomic contracting programs in general, and the 8(a) program in particular, are widely suspected of being exploited by unscrupulous businesses to gain an advantage in competition for federal contracts. Repeated audits over the years have identified many instances of fraud and mismanagement. As an example, the firm receiving the most preferential awards in the dataset compiled here is MicroTechnologies, LLC, of Vienna, Virginia. The firm was an 8(a) participant and carried certifications as a minority-owned business, small disadvantaged business, and service-disabled veteran-owned small business throughout the period examined. Over the period, the firm received $1.19 billion in federal awards, or an average of nearly $200 million per year. By law, firms may not receive more than $100 million in contracts over the course of their participation in the 8(a) program.

While these numbers seem to imply contracting practices well outside the intent of the 8(a) program, the SBA responds that the complicated nature of how the contracts were awarded, the duration of them, and the types of contracts involved make direct comparisons difficult. It is also important to note that awards do not necessarily equate to disbursed funds. Awards may be later de-obligated if the funds are not required for execution of the contract. They do not capture work subcontracted to other firms. Data entry errors are also a consistent problem. However, awards are the only metric reported and must serve as a proxy for disbursements. Below, I examine this assumption through the correlation between awards and short-run firm performance. The relationship is indeed positive, suggesting awards serve as a reasonable proxy, although the modeling below avoids using awards as an explanatory variable of interest.

In 2013, the SBA moved to debar MicroTechnologies, LLC, from future federal contracts, saying it misrepresented its ownership and operational arrangements to receive its preferential contracts. As a condition of continuing to receive government contracts, the CEO was required to step aside. He was allowed to return to the firm in May 2014 after signing a code of ethics and completing contracts compliance training. As of mid-2016, the firm had left the 8(a) program, but continued to receive preferential federal contracts as a service-disabled veteran-owned small business, recording nearly $275 million in awards in the year to April 2016. It claims to be a prime contractor on more than 100 federal projects.

Despite the large award values, the NETS recorded only $6.4 million in firm sales in 2007, rising to $14.1 million in 2013. Funds obligated to the firm may not be ultimately booked as revenue, as described above. Nevertheless, there is clearly a lot going on that requires further explanation. It requires a high level of sophistication to manage 100 federal projects and $275 million in annual awards. That a firm can do so and continue to be classified as “small” and worthy of subsidy speaks to the nature of the regulatory environment. Given the level of scrutiny surrounding the firm after the scandal, it is unlikely that its operations continue to constitute fraudulent behavior; however, it is illuminating to see the extent to which rules designed to help small businesses grow and compete can be turned to the advantage of firms that few impartial observers would consider small or disadvantaged.

At the other end of the distribution are a large number of firms receiving small amounts of contract awards. Figure 4 shows the kernel density function of total awards by firm (left panel) and annual sales by firm as of 2007 (right panel). The median firm received approximately $533,000 in awards over the 6-year period, while the heavily right-skewed distribution continues to $1.19 billion for MicroTechnologies, LLC (not shown in the figure). The median firm by 2007 sales took in $997,000, with the largest firm in the sample, Sprague Operating Resources, LLC, taking in $1.66 billion in 2007 (again, the figure is truncated and does not show this firm). Sprague Operating Resources had 250 employees and received $301,000 in contracts as a woman-owned small business.

Fig. 4
figure 4

Kernel density functions of awards and sales by firm. The median firm in the sample received $533,000 in awards between 2007 and 2012 and had $997,000 in 2007 sales. The tails of the kernel density distributions continue to the maximum values of $1.2 billion in awards (left panel) and $1.7 billion in sales (right panel), not shown here. The figure includes all firms in the sample that began the period with positive employment and sales (see Table 3, column 1)

Full size image

An important question is the viability of the many small firms in the sample. Are these going concerns, short-lived start-ups, or perhaps little more than shell companies used to take best advantage of the regulatory environment? Figure 5 illustrates descriptive statistics for the firms in the dataset segmented by Dun & Bradstreet credit rating. These credit ratings range from one (high) to four (limited). The top panel shows the number of firms in each credit rating category in each year of the sample. Approximately half of the firms did not receive a credit rating. Absence of a rating does not imply lack of viability, but only that Dun & Bradstreet did not have sufficient information to classify the company, whether for lack of historical data, a deficit net worth, or lack of sufficient payment information (all common conditions for young businesses). Of those that did receive ratings, only a small number received the lowest (4) and highest (1). Despite most of the firms in the sample not being rated, those that are rated received substantially larger contract awards, as shown in the center panel of Fig. 5. The “high” credit category received the highest average amount of awards. The bottom panel of Fig. 5 shows average employment levels by credit category. As expected, firms without a credit rating tended to have low levels of employment, with an average in 2007 of only 10.6 employees. The largest firms by employment were those with the highest credit ratings, although there is no correlation of employment with credit ratings for the lower three credit categories. To control for this wide variability in credit scores and the implied access to capital, the models below include a set of dummy variables coded to one for firms that had the highest credit score in each year of the sample. Other specifications which included dummy variables annotating firms with no credit scores (not reported here) produced similar results.

Fig. 5
figure 5

Firm characteristics by credit rating and year. The top panel depicts the total number of firms in each Dun & Bradstreet credit rating category in each year of the sample. The center panel shows average contract awards by credit rating, and the bottom panel shows mean employment levels. The table includes all firms in the sample that began the period with positive employment and sales (see Table 3, column 1)

Full size image

I present summary statistics of variables captured in the FPDS and NETS datasets in Table 4. There are 7141 small businesses in the dataset that received federal funding between fiscal years 2007 and 2012 and did not belong to any socioeconomic category examined here. Each of these firms received an average of $4 million in awards over the period (or about $670,000 per year). They began with an average of 23.3 employees, which fell to 21.6 employees by 2013. Annual sales fell from an average of $4.6 million to $3.6 million. They were 24.6 years old on average as of 2013. The non-preferred firm receiving the most awards over the 6-year period ($1.03 billion) was Energy Enterprise Solutions, LLC, (also known as 1Source) of Germantown, Maryland, in the suburbs of Washington, DC. The firm is a minority-owned small business but is not classified as either disadvantaged or otherwise eligible for contract preferences. The firm’s annual awards over the period examined varied between $39.4 million and $243 million. Total sales over the period ($467 million) amounted to less than half the firm’s reported awards. An objective of the econometric procedures in the models to follow is to control for the effects of such questionable data points. Matching on total awards ensures treatment groups are balanced with similar control observations, and dummy variables indicate firms with total awards summing to more than reported sales (there are 3681 of these in the sample).

Table 4 Summary statistics by socioeconomic category
Full size table

Small disadvantaged businesses make up 6713 firms in the sample with mean awards of $18.8 million and 22.6 employees. The sample contains 1621 service-disabled veteran-owned businesses; these tend to be substantially younger, at 16.2 years, and received $21.6 million in awards on average. Differences in the sample of 8(a) firms are apparent in the summary statistics; there are 3738 of them in the dataset and they have the highest level of average awards by a wide margin, at $26.6 million. Remarkably, their average contract awards are 6.5 times greater than for non-preferred firms, but they report lower average annual sales. A similar (but smaller) trend is apparent for small disadvantaged businesses and service-disabled veteran-owned businesses, although much of this is driven by overlap with the 8(a) category. The sample of 3005 firms participating solely in the 8(a) program is the only one with mean total awards over the period ($25.0 million) outstripping mean total sales ($23.7 million). This trend suggests much higher levels of subcontracting activity. Analysts have long suspected the 8(a) program is used in this way by established firms to capture non-competitive awards; this evidence supports that theory.

3.2 Empirical approach

I use several distinct modeling techniques to assess the degree to which the 8(a) program encourages growth in participating firms. In all cases, the unit of analysis is the individual firm. The first model examines the full set of 16,796 firms which began the period with positive employment and sales (see Table 3, column 1). Using a probit model, it assesses the probability of a firm remaining in business over 6 years for each of the target socioeconomic classifications. The model is specified as

$$ {oob}_i=\alpha +\beta {x}_i+\gamma {y}_i+\delta {Z}_i+{\varepsilon}_i $$

where oob i is a dummy variable coded to one for firms that went out of business between January 2007 and January 2013. The variable x i is a dummy variable indicating program participation for 8(a) firms, while y i indicates service-disabled veteran-owned small businesses. The coefficients of interest are β and γ. The matrix Z i contains control variables including status as small disadvantaged businesses (the superset of 8(a) firms) and veteran-owned businesses (the superset of service-disabled veteran-owned businesses). These variables control for business owner characteristics. Other control variables include the natural logarithms of awards received, total employment at the beginning of the period, firm age, and the number of other firms of the same 8-digit SIC and core-based statistical area. The latter variable controls for agglomeration effects of localized ecosystems of firms that implicitly or explicitly share information, expertise, and perhaps employees. Participants in the Small Business Innovation Research Program, which supports small entrepreneurs, are also indicated with a dummy variable as the nature of these businesses is substantially different, with a heavy focus on research and development. Another dummy variable annotates firms with total awards exceeding reported sales. Effects of access to capital are controlled through four dummy variables indicating firms with credit scores of 1 in each of the first 4 years of the period; Fig. 5 illustrates that these firms represent a small minority but capture an outsized share of awards. Similar dummy variables are not included for 2011, 2012, or 2013 since they are near perfect predictors of a firm remaining in business through 2013. Error is captured in ε i and is assumed orthogonal.

The next model examines only the 14,978 firms that remained in business through 2013 (see Table 3, column 2) and uses ordinary least squares (OLS) to estimate the impact of program participation on firm-level employment and sales growth between 2007 and 2013. I model the relationship as

$$ {\mathrm{growth}}_i=\alpha +\beta {x}_i+\gamma {y}_i+\delta {Z}_i+{\varepsilon}_i $$

where growth i is log growth in employment or sales for firm i between 2007 and 2013. Growth is calculated according to the formula (value2013−value2007)/value2007. The treatment dummies x i and y i again indicate participation in each of the socioeconomic programs, with β and γ the coefficients of interest. The matrix of control variables, Z i , is similar to the probit model above with two exceptions. The model of employment growth uses 2007 sales as a scale variable, while the sales growth model uses 2007 employment. The model also includes the full set of high-credit dummy variables which indicate credit scores of 1 in each year of the sample. Robust standard errors are clustered by 6-digit NAICS code.

To test the robustness of the OLS model and more carefully account for the possibility of omitted variable bias, I next examine performance of 8(a) participants using a difference-in-difference specification directly comparing levels of employment and sales of 8(a) participants and non-participants at the beginning and end of the six-year period. 8(a) treatment firms are first matched to control firms from a select alternative socioeconomic category according to their total level of federal awards between 2007 and 2012. This results in treatment groups of 8(a) firms and equally sized control groups of non-favored, small disadvantaged or service-disabled veteran-owned businesses that received comparable levels of federal funding. The model is specified as

$$ {\mathrm{size}}_{i\tau}=\alpha +\beta {t}_i+\gamma {a}_{\tau}+\delta {t}_i{a}_{\tau}+{\varepsilon}_{i\tau} $$

where size iτ is the log level of employment or sales for firm i in period τ, t i is a dummy variable indicating treatment status of firm i, a τ is a dummy variable indicating either 2007 or 2013 values of the dependent variable, and ε iτ is an error term that is assumed orthogonal. The coefficient of interest is δ, which is the difference-in-difference estimator

$$ \widehat{\delta}=\left({\overset{-}{y}}_{\left(\mathrm{treat}\right)\left(\mathrm{after}\right)}-{\overset{-}{y}}_{\left(\mathrm{treat}\right)\left(\mathrm{before}\right)}\right)-\left({\overset{-}{y}}_{\left(\mathrm{control}\right)\left(\mathrm{after}\right)}-{\overset{-}{y}}_{\left(\mathrm{control}\right)\left(\mathrm{before}\right)}\right) $$

Robust standard errors are again clustered by 6-digit NAICS code. Alternative specifications containing various control variables (not shown) did not substantially alter the results, implying that the difference-in-difference specification adequately accounts for omitted variable bias.

A final set of robustness tests uses propensity score matching to generate control groups from among the group of non-8(a) firms belonging to the socioeconomic category of comparative interest. This method allows matching along a wide array of firm characteristics beyond simply total federal contract awards (as applied in the difference-in-difference model above). A probit model of the form

$$ {t}_i=\alpha +\beta X+{\varepsilon}_i $$

estimates the probability that a given firm would be an 8(a) participant (t i ) given the matrix X of independent variables, which includes logarithms of the number of firms in the same city/industry, employment in 2007, sales in 2007, firm age in 2013 and total federal awards to the firm over the period as well as the “awards greater than sales” and credit rating dummies described in the OLS model above. The error term ε i again captures unexplained variation. Estimated coefficients β are used to estimate a probability of 8(a) treatment for each firm, and treated firms are matched to control firms by this value.

After 8(a) program participants are matched to untreated firms (i.e., non-participating contractors from the select socioeconomic category) by propensity score, I estimate the average treatment effect on the treated (ATT) by the formula

$$ E\left[{Y}_{1 i}-{Y}_{0 i}\ |\ {D}_i=1\right]= E\left[{Y}_{1 i}\ |\ {D}_i=1\right]- E\left[{Y}_{0 i}\ |\ {D}_i=1\right] $$

where E[Y 1i  | D i  = 1] is the log value of the treatment group outcome (percent change in employment or annual sales) for firm i given treatment and E[Y 0i  | D i  = 1] is the counterfactual control group outcome assuming treatment. Matched controls supply counterfactual outcome estimates. Robust standard errors in the models employ the methodology of Abadie and Imbens (2012), in which error from the first stage probit matching procedure is accounted for in final estimation of the ATT.

The probit and OLS specifications described above examine both the 8(a) and service-disabled veteran socioeconomic categories in a single model, while the subsequent difference-in-difference and propensity score matching models directly compare 8(a) firms with individual quasi-experimental control groups. To assess the overall effect of 8(a) business development program participation, I first estimate the effect of treatment on firm performance compared to that of baseline non-favored firms. I then estimate it relative to non-8(a) small disadvantaged businesses, which benefit only from the federal spending goal of 5% and not the broader array of 8(a) benefits and SBA administrative support (see Tables 1 and 2). Since 8(a) firms must be small disadvantaged businesses, this comparison assesses the unique impact of 8(a) program participation controlling for business owner characteristics common to both groups. Finally, I compare 8(a) firms to service-disabled veteran-owned small businesses, which benefit from a similar spending goal, set-asides, and sole-source contract authority. This comparison examines in closer detail the nature of the 8(a) program apart from direct monetary benefits for which participants are eligible. It assesses the impact of the SBA administrative environment, including mechanisms for directing government contracts and encouraging firm growth through management assistance and facilitated relationships with larger firms. These program characteristics are of particular interest, and the comparison with service-disabled veteran-owned firms isolates their effects. In combination, these models provide multifaceted and robust evidence of relative performance differences between programs with different characteristics. The following section enumerates these differences in more detail.

4 Results

This section presents the results of the models outlined in Section 3. It begins with the probit model estimating the probability of a firm remaining in business between 2007 and 2013 given its participation in the socioeconomic programs of interest. It then uses OLS to assess the impact of program participation on growth. Difference-in-difference and propensity score matching ATT models substantiate the OLS findings.

4.1 Probability of remaining in business

The results of the probit model estimating the probability of remaining in business for firms in various socioeconomic categories are presented in Table 5. Relative to non-preferred firms, 8(a) participants are no more or less likely to go out of business over the 6-year period examined. The model controls for business owner characteristics through inclusion of a separate dummy variable indicating status as a small disadvantaged business. (as Table 3 shows, there are 2975 small disadvantaged businesses which did not participate in the 8(a) program during the period examined). The coefficient on the small disadvantaged business dummy variable suggests that these firms are nearly 12% more likely to go out of business than non-disadvantaged firms.

Table 5 Probability of exit by socioeconomic program
Full size table

The contrast of 8(a) firm performance with that of service-disabled veteran-owned small businesses is illuminating. Despite veteran-owned firms in general being more likely to go out of business than baseline non-preferred firms, the contract preferences available to disabled business owners make them 22% less likely to go out of business over the period examined. The result has strong statistical significance. The contrast of service-disabled business performance with that of 8(a) participants suggests that the structure of the 8(a) program, apart from the direct effects of subsidies, does not support long-term viability of participating firms. This finding will be examined further below.

4.2 Ordinary least squares

Presented in Table 6, I use ordinary least squares to model the effects of program participation on employment and sales growth for firms which remained in business throughout the period examined. Program participation is indicated by dummy variables as enumerated in column 2 of Table 3. The model tests effects of program participation controlling for characteristics of disadvantaged and veteran business owners. In this model, small disadvantaged businesses and veteran-owned businesses display rates of growth similar to those of non-preferred firms. 8(a) participants and service-disabled veteran-owned businesses, however, experience substantially stronger growth. Given the log dependent variable, I use the transformation

$$ 100\left[{e}^{\beta}-1\right] $$
Table 6 Ordinary least squares with socioeconomic categories
Full size table

to evaluate marginal changes of x i from 0 to 1 (and similarly for γ and y i when evaluating service-disabled veteran-owned businesses) and

$$ 100\left[{e}^{-\beta}-1\right] $$

to evaluate marginal changes from 1 to 0. Both groups of interest experience employment growth approximately 12% higher than baseline firms, even after controlling for the total amount of awards received. Sales growth is more than 20% higher for 8(a) firms and 17% higher for service-disabled veteran-owned businesses.

The OLS models control for differential levels of contract awards and industry clustering, and examination of the coefficients on these control variables also provides interesting insight. Despite the problems associated with using awards as a proxy for disbursed funds, the variable is strongly correlated with increased growth across all specifications. Positive agglomeration effects are visible in the coefficient of the geographic clustering variable. Firms with more peers in the same industry within their local area see significantly stronger growth in employment and sales relative to those which are geographically isolated.

4.3 Individual program comparisons: difference-in-difference

I next turn to examining pairwise relationships between firms participating in the 8(a) program and the comparison groups. These comparison groups here exclude firms belonging to overlapping categories, such as 8(a) firms owned by veterans. Having constructed treatment and control groups of 8(a) and non-8(a) firms that received comparable levels of contract awards, I construct a difference-in-difference model comparing levels of employment and sales before and after treatment (i.e., in 2007 and 2013). Results are presented in Table 7.

Table 7 Difference-in-difference estimates, samples matched by total funding
Full size table

In the comparison of 8(a) participants with baseline non-preferred firms, matching on award levels produces treatment and control groups of 1494 firms each. The results again show statistically significant outperformance of 8(a) firms, with sales growth performance stronger than employment growth performance. There are 1026 non-8(a) small disadvantaged businesses that match with 8(a) firms by award level, and this comparison again shows that 8(a) firms do well. Coefficient magnitudes are comparable to, though somewhat higher than, estimates from the OLS model. Also supporting the findings from the OLS model is the comparison with service-disabled veteran-owned businesses. There are 664 firms each in the treatment and control groups, and the difference-in-difference model shows no substantial difference in performance. In fact, 8(a) firms appear to do somewhat worse than the comparison group, although the levels of statistical significance are weak.

4.4 Individual program comparisons: propensity score matching

A final test of robustness comes from estimation of the ATT using treatment and control groups constructed through propensity score matching. The probit model in Table 8 generates propensity scores used to match treated firms with control firms for estimation of the ATT in Table 9. These models again exclude from consideration firms from overlapping socioeconomic categories. Results are broadly similar to both of the previous models, with coefficients of similar magnitude. 8(a) firms perform substantially better than non-preferred and non-8(a) disadvantaged firms and comparably to service-disabled veteran-owned businesses.

Table 8 Probit model for generating propensity scores
Full size table
Table 9 PS matching estimates of average treatment effect on the treated
Full size table

5 Conclusion

The expressed purpose of the 8(a) business development program is to “improve the conditions of socially and economically disadvantaged groups” by deliberately steering federal contracts to their firms and providing a broad system of business support mechanisms that teach disadvantaged business owners to compete in the marketplace. This support ostensibly encourages firm-level growth and competition among government suppliers which “promotes economy in [federal] procurements.” Whether these objectives will be met hinges on two competing theories. Under the first hypothesis, historical discrimination has left talent underutilized in the population of potential government suppliers. By deliberately counteracting this discrimination, the government can help minorities and other disadvantaged groups overcome biases and grow to their full potential. Under the second hypothesis, regulatory complexity and diminished competition produce rent-seeking and competition along non-productive lines.

Firms that benefit from the wide array of advantages available through the 8(a) program (i.e., socioeconomic contracting goals, sole-source contracts, management assistance, and mentor-protégé relationships) unquestionably outperform non-preferred firms which do not so benefit. However, a distinction must be drawn between the effects of funding and the effects of training and experience. The intent of the 8(a) program is to offer opportunities to disadvantaged business owners who have been excluded from the market by discrimination (or its legacy) and thereby enhance their ability to compete. Preferred funding arrangements are merely a means to the desired end of producing viable businesses. We would expect businesses awarded non-competitive contracts to grow; in the short term, this says nothing about their underlying competitiveness. The important question is whether owners learn from the experience to create stronger businesses or simply take advantage of the preferential treatment to enrich themselves. Do they create value, or do they leave that to market incumbents and simply capture rents as middlemen for non-competitive contract awards?

This investigation disentangles the effects of funding from the effects of training by using carefully selected comparison groups. First, examining the performance of 8(a) participants relative to non-preferred baseline and non-8(a) disadvantaged firms illuminates the total effect of program participation. Examining their performance relative to service-disabled veteran-owned businesses controls for preferential funding arrangements and isolates the effect of training and business experience gained through 8(a) participation.

The results can be summarized thus: 8(a) firms are no more likely than non-preferred firms to remain in business over a 6-year period, and they are substantially less likely than service-disabled veteran-owned businesses to do so. They grow faster than non-subsidized firms, but not relative to a control group that receives comparable preferential contracting treatment. All the benefits of the program therefore appear to be generated by the short-run effect of subsidies. Participating firms outperform because they are being handed more money, not because they are becoming stronger businesses.

The broad array of support mechanisms available through the SBA (apart from preferential funding) appears to have no effect on the growth of program participants and may even have detrimental effects on long-run firm viability. Service-disabled veteran-owned small businesses, which do not receive them, grow at similar rates and are more likely to remain in business. Although it deserves greater investigation, this may be caused by the limited term of the 8(a) program and the effect of front companies identified by Bates and Williams (1995). Rather than taking advantage of the training and experience to create strong businesses, participants take advantage of preferential contracts while they can and then exit the market (or perhaps re-incorporate) once the benefit ends. The finding above that unusually high levels of 8(a) contract awards are passed on as subcontracts also supports this hypothesis. Rather than being an incubator for new competitors in the market, the program appears to support short-run rent seeking by preferred classes of citizen, who take advantage of the regulatory environment to capture a portion of the profits from industry incumbents which perform much of the work.

Of the competing theories of program effects, the rent-seeking hypothesis receives greater support. Providing more benefits to disadvantaged groups via the 8(a) program does not produce the experience-driven growth premium predicted by the “money on the table” model. Long-run viability is in fact inhibited relative to other socioeconomic groups receiving comparable monetary benefits. The findings here support a growing body of anecdotal evidence that the program does not achieve its stated objectives.

These findings have clear policy implications. Above all, they point to adverse effects of the complex regulatory environment. A substantial bureaucratic infrastructure exists to support the 8(a) program. In models controlling for the relative advantage of set-asides and sole-source awards, as in comparison with service-disabled veteran-owned businesses, the multitude of rules surrounding the program has a sharply adverse effect on long-run viability. This implies the system set up to provide management assistance, mentorship, and other such benefits is not functioning as designed and re-examination of the regulatory structure is necessary. The limited term of the program, rather than encouraging firms to become self-sustaining, in fact encourages them to exit the market once subsidies are withdrawn.

As previous authors have found, program design is crucial in determining whether a given affirmative action program helps or hinders subsidized groups in the long run. Policy makers may be able to improve the functioning of the 8(a) program by simplifying the regulatory environment and re-examining the system of support provided by SBA administrators. Simplification would make questionable behavior more transparent and perhaps encourage the desired types of business owners (those intending to generate growth through value creation rather than rent seeking) to enter the program. This analysis suggests, however, that caution is in order. Lawmakers are not operating in a static market environment. As they have in the past, rules designed to fix today’s problems may only create new opportunities for rent seeking.