The usefulness of accounting estimates for predicting cash flows and earnings

Abstract

Accounting estimates and projections potentially improve the relevance of financial information by providing managers a venue to convey to investors forward-looking, inside information. The quality of financial information is, however, compromised by the increasing difficulty of making reliable estimates and forecasts and the frequent managerial misuse of estimates. Given the ever-increasing prevalence of estimates in accounting data, particularly due to the move to fair value accounting, whether these opposing forces result in an improvement in the quality of financial information is among the most fundamental issues in accounting. We examine the contribution of accounting estimates embedded in accruals to the quality of financial information, as reflected by their usefulness in the prediction of enterprise cash flows and earnings. Our out-of-sample prediction tests indicate that accounting estimates beyond those in working capital items (excluding inventory) do not improve the prediction of cash flows. Estimates do, however, improve the prediction of next year’s earnings, though not of subsequent years’ earnings. We conclude that the usefulness of accounting estimates to investors is limited and provide suggestions for improving the usefulness of estimates.

1 Introduction

Financial statement information—be it balance sheet items such as net property, plant, and equipment; goodwill and other intangibles; accounts receivable and inventories; or key income statement figures such as revenues, pension expense, in-process R&D, or the recently expensed employee stock options—is largely based on managerial estimates and projections. The economic condition of the enterprise and the consequences of its operations as portrayed by quarterly and annual financial reports are therefore an intricate and ever changing web of facts and conjectures, where the dividing line between the two is largely unknown to information users. With the current move of accounting standard-setters in the United States and abroad toward increased fair-value measurement of assets and liabilities, the role of estimates and projections in financial reports will increase.

What is the effect of the multitude of managerial estimates embedded in accounting data on the usefulness of financial information? The answer is far from straightforward. On the one hand, estimates and projections are potentially useful to investors because they are the primary means for managers to convey credibly forward-looking proprietary information to investors. Thus, for example, the bad debt provision, if estimated properly, informs investors on expected future cash flows from customers; restructuring charges predict future employee severance payments and plant closing costs; and the capitalized portion of software development costs (SFAS 86) informs investors about development projects that passed successfully technological feasibility tests and are accordingly expected to enhance future revenues and earnings.¹ This potential contribution of managerial estimates to investors’ assessment of future enterprise cash flows underlies the oft-quoted statement by the Financial Accounting Standard Board (FASB) in its Conceptual Framework about the superiority of accruals earnings—mostly based on estimates—over the largely fact-based cash flows in predicting future enterprise cash flows: “Information about enterprise earnings based on accruals accounting generally provides a better indication of an enterprise’s present and continuing ability to generate favorable cash flows than information limited to the financial aspects of cash receipts and payments (FASB 1978, p. IX).”

On the other hand, the contribution of estimates to the usefulness of financial information is counteracted by two major factors:

• 1. Objective difficulties. In the increasingly volatile business environment, due to fast-changing market conditions (deregulation, privatization, emerging economies) and rapid technological changes, it is difficult for managers to make reliable projections of business events. Consider, for example, the estimated future return on pension assets—a key component of the pension expense: this estimate is essentially a prediction of the long-term performance of capital markets. Are managers better predictors of market performance than investors?² Or reflect on the generally large impairment charges of fixed assets and acquired intangibles (including goodwill) mandated by SFAS 144 and SFAS 142: the determination of these charges requires managers to estimate future cash flows from tangible and intangible assets. In today’s highly competitive and contested markets, the reliability of asset cash flows forecasted over several years is obviously questionable. Accordingly, the accounting estimates and projections underlying financial information introduce a considerable and unknown degree of noise, and perhaps bias, to financial information, detracting from their usefulness.

• 2. Manipulation. Add to the above objective difficulties in generating reliable estimates the expected and frequently documented susceptibility of accounting estimates to managerial manipulation; and the consequent adverse impact of estimates on the usefulness of financial information becomes apparent. Given that it is very difficult to “settle up” with manipulators of estimates—even if an estimate turns out ex post to be far off the mark, it is virtually impossible to prove that ex ante the estimate was intentionally manipulated—there are currently no effective disincentives for managers to manipulate accounting estimates. Indeed, many of the Securities and Exchange Commission (SEC) enforcement cases alleging financial reporting manipulation concern misuse of estimates underlying accruals (for example, Dechow et al. 1996).

Thus, the impact of estimates underlying accounting measurement and reporting procedures on the usefulness of financial information is an open question. The relevance of this examination cannot be overstated. Accounting estimates and projections underlie many Generally Accepted Accounting Principles (GAAP) and consume most of standard-setters’ time and efforts. Just consider the major issues addressed by the FASB in recent years—financial instruments, employee stock options, fixed assets and goodwill impairment, and the valuation of acquired intangibles, to name a few—all require major estimates and forecasts in the process of accounting measurement and reporting. If these and other accounting estimates do not contribute significantly to the usefulness of financial information, the efforts of accounting regulators, and even more importantly, the resources society devotes to the generation of estimates in the process of financial statement preparation and their auditing, are misdirected. Worse yet, if financial information users are led by the estimates-based accounting information to misallocate resources, an additional dead-weight cost is imposed on society.

We define and test the usefulness of estimates embedded in accrual earnings in terms of their ability to predict enterprise performance.³ This predictive use of financial information is central to security analysis and valuation and is also a fundamental premise of the FASB’s Conceptual Framework as indicated by the quote above. Future enterprise performance is mainly reflected by cash flows and earnings. Future cash flows are at the core of asset and liability valuation rules. Thus, for example, asset impairment (SFAS 144) is determined by expected cash flows, and the useful lives of acquired intangibles (SFAS 142) are a function of future cash flows. More fundamentally, enterprise cash flows are postulated by economic theory as the major determinants of asset values. Given a certain ambiguity about the specific definition of cash flows used by investors, we perform our tests with two widely used and frequently prescribed cash flow constructs: cash from operations (CFO) and free cash flows (FCF).

Despite the prominence of cash flows in economic asset valuation models, there is no denying that many investors and analysts are using financial data to predict earnings. The underlying heuristics are somewhat obscured; perhaps investors predict earnings first and derive future cash flow estimates from the predicted earnings. In any case, earnings prediction is prevalent in practice, and we therefore also examine the usefulness of accounting estimates for the prediction of earnings, both operating and net income.

The focus of this study is on accounting estimates, but many of the estimates underlying financial information are not disclosed in the financial reports.⁴ We, therefore, focus in this study on accruals, most of which are based on estimates, and distinguish between accruals that are largely unaffected by estimates (changes in working capital items, excluding inventory) and accruals that are primarily based on estimates (most nonworking capital accruals and inventory). This enables us to draw sharper inferences on the effect of estimates on the usefulness of financial information. We also analyze a smaller sample of firms with data on specific estimates, which we split into recurring and nonrecurring estimates to separate noise (the nonrecurring estimates) from information (the recurring estimates).

Our empirical analysis is based on a sample of all nonfinancial Compustat firms with the required data—ranging from roughly 1,500–3,200 companies per year—spanning the period 1988 through 2005. Our tests are conducted in three stages:

• 1. In-sample, industry-by-industry, predictions of future enterprise cash flows and earnings, based on (a) current cash flows only (the benchmark), (b) earnings, and (c) the set of cash flows, the change in working capital (excluding inventory), and various components of accruals based on estimates. Here we follow the regression procedures of Barth et al. (2001) and find, on more recent data, results that are generally consistent with theirs. This is our departure point.

• 2. Given that in-sample regressions are not prediction tests, we perform out-of-sample firm-specific predictions of future cash flows and earnings, using the industry specific parameter estimates of the in-sample regressions. The focus of this analysis is on the improvement in the quality of predictions brought about by the addition of estimates (accruals) to the predictors. Our results show that accounting estimates do not improve the prediction of future cash flows (both operating and free cash flows), compared with predictions based on current cash from operations and the change in working capital excluding inventory. Notably, cash flow predictions based on current earnings only are significantly inferior to those generated by current cash from operations, contrary to Kim and Kross (2005). In our small sample analysis, neither recurring nor nonrecurring estimates improved significantly the predictions of either cash flows or earnings. The bottom line: accounting estimates beyond those in working capital items (except inventory) do not improve the prediction of cash flows. However, accruals do improve next year’s prediction of net and operating income.

• 3. Finally, we examine the economic significance of estimates. These tests complement stage two, which is based on the statistical significance of differences in the quality of alternative predictors. We perform various portfolio tests, where portfolios are constructed from predicted cash flows and earnings based on various predictors, some of which are based on estimates. The abnormal returns on these portfolios, generated by alternative predictors, are our gauge of economic significance. The focus here is on comparing the returns on portfolios constructed from predictions based on current cash flows only (the benchmark), with returns on portfolios constructed from predictions based on current earnings or on current cash flows plus changes in working capital items and estimates. The results from these tests generally corroborate the out-of-sample prediction tests. In practically all our portfolio tests, the model that uses current operating cash flows only to predict firm performance generates higher abnormal returns than models that add estimates to the prediction process used for the portfolio formation. Furthermore, the portfolios constructed from predictions based on current cash flows only yield abnormal returns with generally lower standard deviation than the alternative portfolios that include earnings or estimates among the predictors.

We caution against sweeping conclusions. We examine the usefulness of accounting estimates in terms of predictive ability with respect to future firm performance. Accounting information is used for other purposes too (contracting, national accounting), for which estimates may be useful. Furthermore, our prediction tests are based on fairly simple models. Users may be using different, more sophisticated models where estimates could prove to be useful. Nevertheless, we believe that our findings draw attention to the significant vulnerability of financial information from the multitude of underlying estimates and projections and to the urgent need for improving the reliability of estimates, on which we comment in the concluding section.

Concerning related research, a substantial number of studies examine the contribution of accruals to the prediction of future cash flows and other variables. These studies can be roughly classified into regression-based (in-sample) analyses, and out-of-sample prediction tests. An example of the former is Barth et al. (2001), who regress cash from operations on lagged values of cash from operations and components of accruals, reporting (p. 27) that “each accrual component reflects different information relating to future cash flows … [and] is significant with the predicted sign in predicting future cash flows, incremental to current cash flows.”⁵ Relatedly, Subramanyam and Venkatachalam (2007) conclude that operating cash flows are more strongly associated with future cash flows than earnings and that current earnings are more strongly associated with future earnings than cash flows. By and large, the in-sample regression studies suggest that accruals are associated with subsequent cash flows and contemporaneous equity values. However, as is argued in Sect. 4.1, in-sample regressions are not prediction tests and may even provide misleading inferences concerning prediction power.

An early out-of-sample prediction test is Finger (1994), who concludes from a sample of 50 companies with much historical data that cash flow is marginally superior to earnings for short-term predictions and performs similar to earnings in long-term cash flow predictions. However, time-series and cross-sectional out-of-sample short-term prediction tests by Lorek and Willinger (1996) and Kim and Kross (2005) show that current earnings predict more accurately future cash flows than current cash flows do. Barth et al. (2005, p. 5) “… find evidence of some reduction in mean prediction errors from disaggregating earnings into cash flows and total accruals, and some additional reduction from disaggregating total accruals into its four major components … median prediction errors generally support disaggregation of earnings only into cash flows and total accruals.” Overall, these findings vary considerably by industry and appear to indicate a more consistent success for the cash flows and total accruals model than for the cash flows and disaggregated accruals model.⁶ Thus, a mixed picture emerges from the out-of-sample tests.

The lack of convergence of the accruals’ usefulness research makes it difficult to draw firm conclusions. Some studies are in-sample, while others are out-of-sample. Some researchers relate accruals to contemporaneous returns or equity values, while others relate them to future values. Some predict cash flows, while others predict equity values based on models using forecasted or realized residual earnings. Our main contribution is the focus on the estimates embedded in accruals and the provision of certain closure to the usefulness of accruals issue. We distinguish between accruals that are largely based on facts and those primarily reflecting estimates, to focus on the usefulness of accounting estimates. Our main tests are out-of-sample predictions, replicating what most investors actually do—predict, with no ex post information (as implicitly assumed by in-sample studies), various versions of future earnings and cash flows. The comprehensiveness of our predicted performance measures (two versions of earnings and two of cash flows) and the number of future periods examined (years t+1, t+2, and aggregate next 2 years and next 3 years) enables us to draw general conclusions about the contribution of estimates to firm performance prediction. Furthermore, our study is the first to examine both the statistical and economic performance of accruals-based prediction models.⁷ The focus on accounting estimates, the out-of-sample methodology, and the examination of both statistical and economic significance—all bringing certain closure to the research question—are our main contributions.

The order of discussion is as follows: Sect. 2 outlines our research design, while Sect. 3 describes our sample. Section 4 reports our prediction tests’ results, and Sect. 5 explains our robustness checks. Section 6 focuses on a subsample with an extended set of accounting estimates. Section 7 reports our portfolio (economic significance) tests, while Sect. 8 concludes.

2 Research design

Our research design consists of three stages: (a) in-sample association tests of cash flows (earnings) regressed on lagged values of these variables and accruals, (b) out-of-sample forecasts of cash flows (earnings) based on these variables and accruals, and (c) calculation of hedge future excess returns on portfolios constructed from the out-of-sample predicted cash flows (earnings) in stage (b). For the out-of-sample tests, we use several prediction constructs, primarily to distinguish between accruals largely based on facts and those based on estimates. At one extreme of the accruals disaggregation, we classify all the accruals in the “operations” section of the cash flow statement into working capital changes excluding inventory (ΔWC*) and the remaining accruals, termed “estimates” (EST): Working capital items with the exception of inventory, such as accounts payable and short-term marketable securities, are generally not materially impacted by managerial estimates,⁸ while most of the remaining accruals are in fact pure estimates (for example, depreciation and amortization, bad debt provision, in-process R&D).

At the other end of the accruals disaggregation, we separate out the change in inventory (ΔINV) from the aggregate estimates (EST), given the evidence (for example, Thomas and Zhang 2002) that much of the accruals anomaly resides in inventory, probably due to intentional and unintentional misestimations of this item. We further break out depreciation and amortization and deferred taxes from other estimates because the identification of these items is possible from Compustat data over the entire sample period. This disaggregation is depicted thus:

The various components of accruals along with cash from operations,⁹ depicted in the two exhibits above are the independent variables in the estimation models underlying our in-sample predictions. We add to these variables the cash flow statement figure of capital expenditures (CAPEX), since the dependent variables in our models are future cash flows or earnings, which are generally affected by current investment (capital expenditures). We believe that the addition of capital expenditures to the regressors improves the specification of the in-sample prediction models and sharpens our focus on the relative performance of the accruals components, our focus of study. Indeed, the capital expenditures variable is statistically significant in most of our annual in-sample predictions models.¹⁰

2.1 Prediction tests

Our prediction tests take the following general form. We predict two versions of cash flows (cash from operations and free cash flows) and two constructs of earnings (net income before extraordinary items and operating income) in years t+1 and t+2, as well as in aggregate years t+1 and t+2, and t+1 through t+3. To gain insight into the usefulness of estimates in predicting firm performance, we use five prediction models with increasing disaggregation of accruals (regressors):

• Model 1: current CFO only—the benchmark model;

• Model 2: current net income (NI) only;

• Model 3: current CFO and the change in working capital items excluding inventory (∆WC*)—namely, largely fact-based regressors;

• Model 4: current CFO, the change in working capital items excluding inventory ∆WC*, and total remaining accruals, largely based on estimates (EST); and

• Model 5: current CFO, the change in working capital items excluding inventory (∆WC*), the change in inventories (ΔINV), depreciation and amortization (D&A), the change in deferred taxes (DT), and all other estimates (EST*)—the most disaggregated model.

The purpose is to examine whether the gradual addition of components of accruals estimates to current cash flows (the benchmark) improves the prediction of future cash flows or earnings. Increasing the disaggregation of accruals should, in general, enhance the quality of prediction (from model 1 to 5), since the individual accrual components are allowed to have different effects (multiples) on the predicted values. We examine model 2 because the predictor, earnings, is a summary accounting variable that has been extensively investigated for its information content and has been used in most prior studies (for example, Barth, Cram, and Nelson 2001 and Kim and Kross 2005).

We use industry-specific estimated coefficients from each of the above five prediction models to calculate firm-specific predicted values for cash from operations, free cash flows, net income and operating income. We then calculate firm-specific prediction errors as the difference between the actual and predicted values of each variable examined. The following examples of the prediction of next year’s free cash flows (FCF) will clarify our prediction procedures.

• (a) Benchmark Model using CFO only (example for 1990):

  • 1. Estimate cross-sectionally for each 2-digit industry the following regression: FCF(89) = α + βCFO(88) + ε.

  • 2. Predict for each firm in a given 2-digit industry: EFCF(90) = α + βCFO(89), using the previously determined industry specific estimated coefficients.

  • 3. Determine prediction error for each firm in a given 2-digit industry: FCF(90) − EFCF(90).

Here we predict 1990 free cash flows [EFCF(90)] from current cash from operations, [CFO (89)] (and capital expenditures). First, for each 2-digit industry, we regress cross-sectionally free cash flows of 1989 on CFO in 1988 and obtain the estimated coefficients α and β. Those coefficients are then used to predict firm-specific free cash flows (EFCF) in 1990, using the firm’s actual CFO of 1989. Then, a firm-specific prediction error is determined by comparing the firm’s actual 1990 FCF with the predicted one. The same procedure is repeated for every firm and sample year.

• (b) Expanded estimates, Model 5 (example for 1990):

  • Estimate cross-sectionally for each 2-digit industry:

    $$ {\text{FCF(}}89 )= \alpha + \beta_{1} {\text{CFO(}}88 )+ \beta_{2} \Updelta {\text{WC}}^{*} (88) + \beta_{3} \Updelta {\text{INV(}}88 )+ \beta_{4} {\text{D}}\& {\text{A(88)}} + \beta_{5} {\text{DT(}}88 )+ \beta_{6} {\text{EST}}^{*} (88) + \varepsilon $$

The prediction and error determinations are done as in (a) above.

Here we predict the 1990 free cash flows from 1989 cash from operations, capital expenditures, and the disaggregated set of estimates (see second diagram at the beginning of this section). Once more, we run by industry a cross-sectional regression of 1989 free cash flows on the 1988 values of the independent variables, estimating the α and β₁ … β₆ coefficients (and a β₇ coefficient for 1988 capital expenditures). The firm-specific 1990 free cash flows are predicted using these industry-specific coefficients and the actual values of the independent variables in 1989. Computation of the 1990 free cash flows prediction error follows.

The prediction of free cash flows in t+2 follows the earlier procedure with one difference: the cross-sectional estimate (first equation) and the forecast (second equation) now involve a two-year lag (for example, free cash flows in 1990 regressed on cash from operations of 1988). The same procedure is performed for each firm and sample year. The expanded prediction models incorporating disaggregated accruals follow step (b) above. We also predict free cash flows for aggregate years t+1 plus t+2 and t+1 through t+3. These predictions are based on the procedures described above, except that aggregated future free cash flows are substituted for single year free cash flows as left-hand variables in the various models.

To evaluate the quality of the out-of-sample predictions, we compute summary measures of prediction errors derived from the firm- and year-specific estimated errors: the mean and median signed prediction errors indicating the bias in the forecasts, and the mean and median absolute prediction errors which abstract from the sign of the error, indicate forecast accuracy. The firm-specific prediction error in a given year is computed as the realized value of cash flow or earnings minus the predicted cash flow or earnings, divided by average total assets in year t.

2.2 Portfolio analysis

The third stage of our research design is motivated by Poon and Granger (2003, p. 491) who note: “Instead of striving to make some statistical inference, [prediction] model performance could be judged on some measures of economic significance.” We interpret their statement as saying that we should not rely solely on the statistical significance of our prediction errors calculated in stage two but should also examine and perhaps even rely more on measures of economic significance. To gauge the economic significance of the contribution of estimates to the usefulness of financial information, we perform a series of portfolio tests focusing on the incremental stock returns generated by the estimates-based prediction models.

Essentially, we use the out-of-sample predicted values of cash flows (cash from operations and free cash flows) and alternatively of earnings (net income and operating income), obtained in the second stage of our analysis, to form portfolios. Specifically, for each sample year, we rank all firms (across all industries) on predicted firm-specific cash flows or earnings (four rankings: two for cash flows and two for earnings). We then form 10 portfolios from each annual ranking and compute risk-adjusted (size and book-to-market adjusted) returns from holding these portfolios over several future periods. In assessing the performance of the various predictors (CFO, NI, ∆WC*, accruals of estimates), we primarily focus on a zero-investment (hedge) strategy: going long (investing) in the top portfolio—the 10% of firms with the largest (scaled) predicted cash flows or earnings—and shorting (selling) the bottom portfolio—10% of firms with the lowest predicted cash flows or earnings. The abnormal returns on these zero-investment portfolios indicate the economic contribution to investors of using accounting estimates as predictors. Thus, if estimates are useful to investors then portfolios constructed from predictions based on current cash flows and estimates-based accruals should consistently outperform portfolios formed from predictions based on current cash flows only.

It should be noted that if markets are efficient concerning the information in accruals—a big if, in light of Sloan (1996)—and if investors select securities using procedures similar to our industry-based prediction models specified above, then our subsequent portfolio abnormal returns should be roughly zero. Our purpose in these portfolio tests, however, is not to examine market efficiency, but rather to compare the performance of portfolio selection procedures with the estimates-based accruals against similar procedures without accruals (based on past cash flows only). We are thus focusing on the with- and without-accruals comparisons, being indifferent about market efficiency. Stated differently, the comparative abnormal hedge returns across the five prediction models, rather than the statistical significance of those returns, is our focus of analysis.

3 Sample selection and descriptive statistics

We obtain accounting data from the 2006 Compustat annual industrial, full coverage, and research files, and use data from the statement of cash flows because Collins and Hribar (2002) suggest that such data are preferable to accruals derived from the balance sheet. Since reporting a statement of cash flows was mandated by SFAS 95 in 1987, our accounting data span the period 1988–2005.¹¹ In the in-sample regression analysis, each year from 1988 to 2004 is a predictor year (generating the independent variables), while each year from 1989 to 2005 is a predicted year (providing the dependent variables). Thus, 17 in-sample annual regressions are estimated for each industry.

Our sample selection procedure is as follows. We start with 75,571 observations with values for NI, CFO, ∆WC*, ∆INV, D&A, DT, EST, EST* and CAPEX for the current year, year t, and for NI over a three-year horizon, t−1 to t+1. Firms with all fiscal year ends are included. We control for outliers by following the procedures in Barth et al. (2001). Thus, after eliminating the top and bottom one percentile of current NI and CFO, we are left with 73,324 firm-year observations. By excluding observations with market value of equity or sales of less than $10 million, or with share prices below $1, to eliminate economically marginal firms, the number of observations decreases to 51,301. By deleting observations with studentized residuals greater than 3 or less than −3, we are left with 50,288 observations. Since we conduct industry-by-industry in-sample regression analysis we require each industry to have a minimum of 600 observations over the period 1988–2004. This criterion reduces the sample to its final size of 41,124 observations. We obtain stock returns data for the portfolio analysis from the 2006 CRSP files.¹²

Table 1 provides summary statistics (variables are scaled by average total assets) and a correlation matrix for out test variables. Panel A shows that depreciation and amortization (D&A) constitutes the bulk of the estimates underlying accruals (EST): the mean (median) of D&A is 0.054 (0.047), close to the mean (median) of EST, 0.059 (0.052). The mean of net estimates (EST*), excluding D&A and deferred taxes, is quite large, 0.019, and is driven mainly by large positive values, as the median value of 0.004, Q1 of 0.000 and Q3 of 0.019 imply. CFO has the lowest, while NI has the highest variability (standard deviations of 0.129 versus 0.149) among the various earnings and cash flow variables. In panel B all correlations are significant at the 5% level or better. We note the high negative correlations of our estimates variables, EST and EST*, with the income variables, NI and OI. However, the correlations of EST and EST* with both the cash flow variables, CFO and FCF, are much lower; positive for EST and negative for EST*.

Table 1 Descriptive statistics

4 Empirical findings: prediction tests

4.1 Stage one: in-sample regressions

As a departure from available research, we essentially replicate the in-sample regressions of Barth et al. (2001) on recent data, regressing cash from operations on lagged values of cash from operations and earnings components (Model 5 in Sect. 2.1), and find that in each of the 23 industries examined the lagged cash from operations and change in working capital minus inventory are highly significant. In the majority of the industries, inventory change is also significant, as is depreciation and amortization. However, deferred taxes and other accruals estimates are significant in about half of the industries only. These results (available on request) are consistent with Barth et al. (2001).

However, it is important to note that a regression analysis of a given variable on lagged values of that variable along with other data, as frequently conducted in accounting and finance research, is not a conclusive test of predictive ability. As noted in Poon and Granger’s (2003, p. 492) survey:

In all forecast evaluations, it is important to distinguish in-sample and out-of-sample forecasts. In-sample forecast, which is based on parameters estimated using all data in the sample, implicitly assumes parameter estimates are stable through time. In practice, time variation of parameter estimates is a critical issue in forecasting. A good forecasting model should be one that can withstand the robustness of an out-of-sample test, a test design that is closer to reality. In our analyses of empirical findings … we focus our attention on studies that implement out-of-sample forecasts.

This important insight motivates our primary analysis, which focuses on out-of-sample prediction tests.

4.2 Stage two: out-of-sample prediction tests

Table 2 summarizes our main out-of-sample prediction findings. Recall that we predict four key performance indicators: cash from operations (CFO); free cash flows (FCF), defined as CFO minus capital expenditures (CAPEX); net income before extraordinary items (NI); and operating income (OI). There are four prediction horizons: next year, second year ahead, aggregate next 2 years, and aggregate next 3 years. Five prediction models are examined (as discussed and demonstrated in Sect. 2.1), where the predictive (independent) variables are (1) CFO only—the benchmark model, (2) NI only, (3) CFO and the annual change in working capital items excluding inventory (ΔWC*), (4) CFO plus the change in working capital items excluding inventory (ΔWC*), as well as the total remaining accruals (EST), which are largely estimates based, including the change in inventory, and (5) our most disaggregated model: CFO, ΔWC*, the change in inventories, depreciation and amortization, deferred taxes, and all remaining estimates. Current capital expenditure is included as an additional variable in each of the five models.

Table 2 Out-of-sample predictions of cash flows and earnings by current cash flows, earnings, and combinations of accruals

We report in Table 2 four summary statistics for the prediction errors of our five models: the pooled firm-specific mean absolute error (MAER) of each of the five models; the pooled mean signed error, or bias (MER); the mean R ²s from annual regressions of firm-specific actual values of future cash flows or earnings on the corresponding predicted values; and the average over the years of Theil’s U-statistics.¹³ We indicate with an ampersand (&), asterisk (*) or a hash (#) the pooled mean absolute prediction errors (MAER), which are significantly different between Models 1 and 2, Models 1 and 3, and Models 3 and 4, and Models 3 and 5, respectively.¹⁴ We have also computed the sample median signed errors, median absolute errors, and root mean square errors. Results from these errors indicators are very similar to those reported in Table 2. (We comment in the text on the occasional differences.) Below are the main inferences we draw from Table 2, and additional analyses:

• 1. Prediction of cash flows. Considering the prediction of cash from operations and free cash flows—left two quadruples of columns in Table 2—we note that the predictions derived from net income only (Model 2) are always significantly inferior to the predictions based on cash from operations only (Model 1). This is true across the four forecast horizons and the four error summary statistics. For example, in predicting one-year-ahead cash from operations (top left panel), the MAER, MER, and Theil’s U are lower for Model 1 than for Model 2 (0.056 vs. 0.062, 0.001 vs. 0.003, and 0.58 vs. 0.64, respectively), while the R² of Model 1 is higher than that of Model 2 (0.46 vs. 0.37). The difference in the MAERs is statistically significant, as indicted by the ‘&’ sign. This pattern is evident across all eight panels reporting predictions of cash from operations and free cash flows for various horizons. Thus, for one- to three-year forecast horizons, current cash from operations is a better predictor of future cash from operations and free cash flows than current net income. This result is inconsistent with Kim and Kross (2005) findings that in one-year-ahead predictions of cash flows current earnings performs better than current cash flows.¹⁵

  Moving on to Model 3, (predictors: cash from operations and the change in working capital items minus inventory), we note that the CFO and FCF predictions derived from current CFO only (Model 1) under-perform predictions based on current CFO and the change in working capital items excluding inventory, ΔWC*. Thus, the mean absolute errors of Model 3 are significantly lower than those of Model 1 in all CFO and FCF panels, except in the FCF panel for the aggregate next 3 years horizon (bottom FCF panel).¹⁶ The reported R ²s and Theil’s U statistics also indicate the under-performance of Model 1 relative to Model 3. For example, in predicting 1-year-ahead cash from operations (top left panel), the MAER and Theil’s U are lower for Model 3 than for Model 1 (0.054 vs. 0.056 and 0.56 vs. 0.58, respectively), while the R ² of Model 3 is higher than that of Model 1 (0.50 vs. 0.46). Thus, for 1–3-year forecast horizons, the total change in working capital items excluding inventory is incrementally informative over current cash flows. This is relevant for our focus on the usefulness of accounting estimates, because the working capital items, excluding inventory, and with the exception of accounts receivable, are largely free of estimates.

  We now move to examine the contribution of accounting estimates to cash flow prediction. We do this by comparing the performance of Models 4 and 5 with that of Model 3, where Model 3 becomes our benchmark given its superior performance up to this point. We note that CFO and FCF predictions derived from Model 4 (based on CFO, the change in working capital items excluding inventory (ΔWC*), as well as all other accruals including the change in inventory) and Model 5 (based on CFO, ΔWC*, the change in inventories, depreciation and amortization, deferred taxes, and all remaining accruals) equally perform or under-perform the predictions from Model 3 (based on CFO and ΔWC*). Specifically, the mean absolute errors of Model 3 are significantly lower than or equal to the mean absolute errors of Models 4 and 5 in all the CFO and FCF panels. Furthermore, the reported MERs, R ²s, and Theil’s U statistics are also consistent with the under-performance of Models 4 and 5 relative to Model 3. For example, in predicting one-year-ahead cash from operations (top left panel), the MAER, MER, and Theil’s U for Model 3 are either equal to or lower than for Models 4 and 5 (0.054 vs. 0.054 and 0.055; 0.001 vs. 0.002 and 0.002; and 0.56 vs. 0.57 and 0.57, respectively), while the R ² of Model 3 is equal to or higher than the R ²s of Models 4 and 5 (0.50 vs. 0.50 and 0.49). Accordingly, we conclude that for 1–3-year forecast horizons the accounting estimates embedded in accruals, either as a lump sum or disaggregated, do not improve cash flow predictions over current cash from operations and the change in working capital (excluding inventory).¹⁷

  Conclusions: Neither total earnings nor disaggregated estimates-based accruals systematically improve the prediction of cash flows (CFO or FCF) over the predictions based on current CFO and the change in working capital items (excluding inventory). This finding is inconsistent with the FASB’s conceptual stipulation that “Information about enterprise earnings … generally provides a better indication of an enterprise’s present and continuing ability to generate favorable cash flows than information limited to the financial aspects of cash receipts and payments” (FASB 1978, p. IX), though our data start 10 years after this statement was issued.

• 2. Prediction of earnings. The two quadruples of columns to the right of Table 2 report prediction performance statistics for net income (NI) and operating income (OI). Here, the predictions derived from net income (Model 2) significantly outperform those based on cash from operations only (Model 1), for the one-year-ahead forecasts. For example, in predicting next year’s operating income (top right panel), the MAER of Model 2 is significantly lower than that of Model 1 (0.057 vs. 0.061). The R²s and Theil’s Us confirm the stronger performance of Model 2, for one-year predictions. Interestingly, Model 2’s predictions are significantly inferior to Model 1’s in the 2-years-ahead and aggregate next 3 years predictions (second and bottom NI and OI panels). For example, in predicting aggregate 3-years-ahead operating income (bottom right panel), the MAER of Model 2 is significantly higher than that of Model 1 (0.257 vs. 0.253). Thus, for a one-year-ahead forecast horizon, current net income is a better predictor of future net income and operating income than current cash from operations.¹⁸

  Of the five models examined for earnings predictions, the best performer is Model 4—with three variables: CFO, ΔWC* (change in working capital excluding inventory), and EST (all other accruals)—for all forecast horizons. Intriguingly, Model 5, where EST is disaggregated to several estimates-based accruals, is somewhat inferior to Model 4. Apparently, predicting from disaggregated accruals results in noisy forecasts.

  Conclusions: Earnings is a better predictor of next year’s earnings than cash flow. Accounting accruals, when disaggregated to working capital items and other accruals, improve the prediction of operating and net income. No further improvement is achieved from a finer disaggregation of accruals.

5 Robustness checks

• 1. How good are our prediction models? Our models are admittedly simple—they obviously abstract from many of the complexities of real security analysis. Nevertheless, the R²s in Table 2—derived from annual regressions of actual values (future cash flows or earnings) on predicted values—are quite large. Thus, for example, for next year’s predictions (top panels of Table 2), the R² range is 0.33–0.58. As expected, the R²s drop for second year predictions, yet they are still in the reasonable range of 0.21–0.37. Thus, despite their simplicity, our prediction models perform reasonably well.

• 2. Trimming extreme prediction errors. The results of Table 2 are after trimming the top 2% of the absolute forecast errors. We also computed prediction errors after trimming the top and bottom 1% of the forecast errors and without any trimming. The resulting patterns of prediction errors (not reported) are in both cases very similar to those of Table 2. As expected, Table 2 trimmed errors are substantially smaller than the nontrimmed errors, the R²s are larger, and the Theil’s U statistics are lower, yet our conclusions regarding the relative performance of the five models equally apply to the nontrimmed errors. Thus, outliers do not affect substantially our inferences.

• 3. Classification by size of accruals. Since the estimates we examine are components of total accruals, we classified the sample firms into three groups, by the size of accruals, to check whether accruals size affects our findings. Specifically, for each sample year we ranked the firms by the size of total accruals (scaled by total assets) and then formed three groups: the top 25% of firms (high accruals), the middle 50% (medium accruals), and the bottom 25% (low accruals). We then generated cash flow and earnings predictions for each of the three accruals groups in the same manner used for the total sample. The findings for all three accruals groups are essentially the same as those for the total sample: accounting estimates do not improve the prediction of cash flows (either CFO or FCF)¹⁹ and do improve the prediction of next year’s earnings. We note a pattern in the accruals classifications: for firms in the medium accruals category (middle 50% of the accruals ranking), the average MAERs are substantially lower and the R²s higher than the corresponding statistics of firms with large or small accruals. Thus, for example, in predicting next year’s net income, the MAER range of the firms in the top accruals quartile is 0.078–0.083 (for the five prediction models), while the corresponding MAER range for the medium accruals firms is 0.043–0.45 only. Thus, accruals, both high and low, adversely affect the performance of all our prediction models.

• 4. Industry effects. To examine whether the contribution of estimates to the prediction of cash flows and earnings varies across industries, we analyze our out-of-sample predictions for each of the 23 industries in the sample. This analysis identified several industries where accounting estimates did not improve even the prediction of next year’s earnings, relative to predictions based on CFO only: oil and gas (SIC#13), printing and publishing (27), fabricated metals (34), eating and drinking places (58), and health services (80). It is difficult, however, to find a common denominator to these industries. In addition, we note the expected difference in the predictive performance of all the five models due to stability of demand conditions: for industries with stable demand, such as electric and gas utilities, the MAERs were very low (range: 1.9–2.1%), while for volatile industries, such as software, the MAERs of all the models were relatively high (range: 9.9–10.8%). In general, stability of customer demand over time increases the accuracy of predicted firm performance, relative to unstable industries. Thus, our general finding that estimates do not improve the prediction of cash flows but do improve short-term earnings prediction, generally holds across industries.

• 5. Temporal changes. To examine for temporal changes in the contribution of estimates to the prediction of cash flows and earnings, we split the sample period—1988 through 1994 and 1995 through 2004—and compare the models’ performance across the two subperiods. The main finding standing out is the deterioration in predictive performance of all the five models in the recent period (1995 through 2004) relative to the early one (1988 through 1994). This significant decrease in the prediction performance of our models is probably caused by the general increase in business volatility (for example, Campbell et al. 2001), as well as by the increased manipulation of earnings via estimates. The latter cause (manipulation) is supported by the fact that the deterioration in the predictive performance of our models is substantially smaller for cash flows than for earnings (for example, for aggregate 3 years prediction of CFO, the MAERs of the five models in the early period are roughly 17%, increasing to only 20% in the latter period, while for earnings the increase is from 20 to 30%). As for the contribution of estimates to the prediction of earnings—it is smaller in the recent period than in the early one, likely reflecting once more the management of earnings via estimates which has increased substantially in the 1990s (for example, Lev and Nissim 2006).

• 6. Excluding firms with M&As, discontinued operations and foreign currency translations. Collins and Hribar (2002) show that the errors in estimating accruals are large in the presence of significant mergers and acquisitions, discontinued operations, and foreign currency translation. We accordingly examine the sensitivity of our findings to these effects by excluding firms with such events. We follow Collins and Hribar to identify firms with mergers and acquisitions, using Compustat’s annual footnote code 1. We use Compustat item #66—Discontinued Operations—to proxy for divestitures and Compustat item #150—Foreign Currency Adjustment (Income Account)—to proxy for foreign currency translation. Unlike Collins and Hribar, who used an absolute cutoff of $10,000 for discontinued operations and foreign currency translations, we use the absolute value of the ratio of item #66 or item #150 over item #18 (net income) as cutoff. If the absolute value of the ratio is greater than 10%, we regard those observations as having significant discontinued operations or foreign currency translations. To conserve sample size, we reduce the requirement of 600 observations (used in the main analysis) to 400 observations per each two-digit SIC code. We also control for outliers using the same procedures described in Sect. 3 above. The new sample consists of 29,500 observations in 25 two-digit SIC groups over the period of 1988 through 2004. The results from this reduced sample show that cash flow predictions derived from net income (Model 2) are significantly inferior to predictions based on cash from operations only (Model 1). However, the opposite is true for earnings predictions. Therefore, these results are consistent with those reported in Table 2.

• 7. Longer series of predictors. The cross-sectional estimation of parameters in the first stage of our predictions (demonstrated in Sect. 2.1) uses only the current values of cash flows and accruals as predictors. Analysts often use longer time series of historical data. Accordingly, we have also experimented with cross-sectional estimates based on the last 3 years of data on CFO, NI, the change in working capital excluding inventories, and remaining estimates. However, the predictive quality of these estimates, based on 3 years of historical data, is slightly inferior to those based on current data (reported in Table 2).

6 A sharper focus on accounting estimates

In the above analysis we focused on four components of accounting estimates available from Compustat for the entire sample period: the change in inventories (ΔINV), depreciation and amortization (D&A), deferred taxes (DT), and all other estimates embedded in accruals (EST*). However, in recent years Compustat expanded the information on estimates, allowing us to examine additional estimates: pension expense (#43), estimated doubtful receivables (#67), post retirement benefits (#292), restructuring costs (#377), in-process R&D (#388), stock compensation expense (#398), and asset writedowns (#381), all important estimates underlying financial information. These are components of the other estimates (EST*) in Model 5. To increase the power of our tests, we selected a sample with valid data on all of these estimates in a given year. Such a restriction was satisfied in the years 2001–2004, and the resulting sample consists of 305 firm-year observations.

Table 3 reports the out-of-sample prediction results for this sub-sample of firms with multiple specific estimates. Given the small number of observations in each individual year, we perform the analysis by pooling the data over the 4 years. We report in Table 3 statistics for cash from operations and net income predictions, estimated by seven prediction models: the original five, as in Table 2, and two additional models based on the specific estimates. Model 6 splits the estimates into recurring and nonrecurring items (recurring estimates (RECUR) consist of depreciation and amortization (D&A), deferred taxes (DT), pension expense (PENSION), provision for doubtful receivables (ARDBT), and post retirement benefits (PRB), while the nonrecurring estimates consist of restructuring costs (RSTRCST), in-process R&D (IPRD), stock compensation expense (STKCMP) and asset write-downs (WRTDWN)). Model 7 is the fully disaggregated model: predictors—current cash from operations, change in working capital excluding inventory, change in inventory, and the disaggregated components of recurring and nonrecurring estimates (D&A, DT, PENSION, ARDBT, PRB, RSTRCST, IPRD, STKCMP, and WRTDWN).

Table 3 Out-of-sample 1-year-ahead predictions of cash flows and earnings by current cash flows, earnings, and combinations of accounting estimates for a sample of 305 firm-years with detailed estimates over the period 2001–2004

The splitting of estimates into recurring and nonrecurring items intends to separate prediction noise (the nonrecurring estimates) from information (the recurring estimates). The expectation is that the recurring estimates will improve the prediction of both earnings and cash flows. However, the statistics reported in Table 3 do not support this expectation. The data are consistent with the overall sample (Table 2): (1) Current cash flows generate the best predictions of future cash flows, while current earnings yield the best predictions of next year’s earnings. (2) Models 6 and 7—based on a fine disaggregation of estimates-based accruals—do not improve the prediction of cash flows or earnings over Model 3 (CFO plus the change in working capital items). For this expanded set of identified estimates, as for the entire sample (Table 2), the disaggregation of estimates doesn’t yield improved predictions of cash flows or earnings.

7 Economic significance tests: portfolio returns

To draw inferences about the economic significance of our predictions, we use the one-year-ahead cash flow and earnings predictions whose errors were analyzed above (Table 2). From these predictions we form portfolios based on cash flow or earnings scaled by total assets. Specifically, we rank the sample firms by the predicted cash flow (earnings) and form 10 portfolios from the ranked firms. Portfolios are formed at the end of April in each year, 1990 through 2004. Finally, we compute the hedge 12-month and 36-month abnormal returns from investing (going long) in the top—highest predicted cash flow (earnings)—portfolio and shorting the bottom—lowest performance—portfolio. The returns are computed for each firm starting from May of each year, 1990 through 2004, that is, the portfolios are aligned in calendar time.²⁰ These returns are adjusted for both size and book-to-market factors in the conventional Fama and French (1992) manner. The hedge returns reported below are the means of the returns from the annual rankings, 1990 through 2004, and the t-statistics are based on the standard errors of the annual mean returns. For comparison, we also compute the perfect foresight returns, where the portfolio formation variables of t+1 cash flow or earnings, are the actual numbers for that year, rather than the predictions.

The hedge 12-month and 36-month returns are presented in Table 4 for portfolios based on predicted cash from operations, free cash flows, net income, and operating income. The four panels of Table 4 reveal an identical pattern: the hedge return on portfolios constructed from Model 1 predictions—using current cash from operations only—are higher than the returns from portfolios constructed from predictions based on Models 2 through 5, using earnings and combinations of accruals as predictors. For example, in Panel A (portfolios constructed from cash from operations predictions), Model 1’s 12-month ahead return is 6.8% (16.7% for 36-months), while all the returns from Models 2 through 5 are lower. (The 36-month return of Model 1 is statistically significant at the 0.05 level.) Surprisingly, the predictions based on cash from operations only (Model 1) also yield the lowest standard errors (not reported) of yearly returns (1990 through 2004) of the five models. Thus, Model 1 generates the highest mean returns with the lowest volatility. As expected, the returns of the perfect foresight model (left column)—a benchmark—are all large and significant.

Table 4 Mean 12-months and 36-months-ahead hedge abnormal returns based on the ranking of actual or predicted cash flow from operations (Panel A), free cash flow (Panel B), net income (Panel C), and operating income (Panel D)

Most of the returns in Table 4, except for Model 1’s in Panels A and B, are statistically insignificant. There are several reasons for that. Our significance tests (Fama and MacBeth 1973) are based on 15 years only (1990 through 2004) and exhibit a few very large outliers, both weakening the power of the tests.²¹ More fundamentally, our prediction models are based on information available to investors. If investors are basing investment decisions on models similar to ours, or better, and if markets are efficient, our subsequent periods portfolio returns should be insignificant. However, the efficiency of capital markets with respect to accruals—the focus of our analysis—was challenged by Sloan (1996) and the multitude of subsequent studies. We, therefore, believe that market efficiency per se should not invalidate our portfolio tests. The uniform return superiority of Model 1—constructed from cash from operations only—is consistent with and lends certain support to our extensive out-of-sample prediction tests (Table 2).

Given the somewhat surprising outcome of the portfolio tests, we subjected them to several of robustness checks.

• 1. Reflection of the accruals anomaly? This anomaly documents a unique pattern of stock returns: extreme accruals are negatively associated with subsequent returns. Is this phenomenon affecting our portfolio tests? To address this question we perform the following test. We classified the sample firms in each year to high, medium, and low current total accruals firms: top 25%, middle 50%, and bottom 25% of the yearly ranking by accruals scaled by total assets. We then performed the portfolio analysis reported above for each of the three accruals groups. The accruals indeed affect substantially the returns on the various portfolios, but the relative performance of the portfolios constructed from our five prediction models (CFO only, NI only, CFO, ΔWC*, and various combinations of estimates) is essentially the same as that portrayed in Table 4. The portfolio returns increase with the size of accruals (for example, in Model 1—CFO only—the return for low accruals firms is 3.5%, while the same model yields 4.7% for high accruals firms). Note that the pattern of increasing returns with the size of accruals is opposite to the decreasing returns of the accrual anomaly that is our returns results are the reverse of those to the anomaly. Thus, this test indicates that our portfolio return findings are not driven by the accruals anomaly.

• 2. Individual deciles and quintiles. The portfolio returns presented in Table 4 are based on a zero-investment strategy: long in the top decile of predicted cash flows or earnings and short in the bottom decile. Basing the zero-investment portfolios on quintiles (top and bottom) rather than on deciles does not change our conclusions. Also, an individual examination of the top decile (long) and bottom decile (short) underlying Table 4 does not reveal a particular contribution of either decile to the hedge portfolio returns.

8 Concluding remarks

Managerial estimates and projections are pervasive in accounting measurement and valuation procedures, affecting to an unknown (by investors) degree practically all income statement and balance sheet items. The contribution of these estimates and projections to the quality of financial data is increasingly challenged in the fast-changing and turbulent business environment, which makes it very difficult for managers to generate reliable projections. The quality of financial information is further compromised by the frequent use of estimates to manipulate financial information. What then is the contribution of accounting estimates to the quality and informativeness of financial information?

We investigate this question by evaluating the contribution of accounting estimates embedded in accruals to the prediction of both cash flows and earnings over various horizons. Our battery of tests, consisting of both out-of-sample prediction and investment portfolio analyses, indicate that estimates, in groups or by individual components, do not contribute appreciably to the prediction of cash flows, (cash from operations or free cash flows), yet do improve next year’s prediction of net and operating income beyond current operating cash flows. However, our investment portfolio tests, designed to reveal economically significant contribution, do not corroborate this improvement.

Our findings, however, are subject to several caveats. First, financial data are used in various contexts besides prediction (for example, contracting uses) for which estimates may be useful. Second, our prediction models—the core of our analyses—are admittedly simple. Perhaps estimates are useful in more sophisticated uses of financial information. Third, many accounting estimates are not reported separately in the financial reports, and we obviously could not test their usefulness. These caveats notwithstanding, we believe our findings cast sufficient doubt on the usefulness of accounting estimates to focus researchers and regulators’ attention on the ways to improve their reliability.

The major implication that we draw from our findings is the urgent need to enhance the reliability of accounting estimates, particularly as the FASB and IASB increase the role of estimates in their push to expand fair value accounting. This issue, however, is sparsely discussed by researchers and accounting regulators, and we are not aware of any current policy actions aimed at enhancing the reliability of accounting estimates/projections. There are several promising proposals in the literature, which, in our opinion, deserve further attention and development. One such proposal was advanced by Ijiri (2002), who calls for a separation by income statement line items of forecasts (estimates) from facts. Such a separation will provide users with an important reliability indicator of major income statement items (for example, compare the case where 5% of the firm’s revenues are based on estimates with the much more uncertain case where 25% of revenues are based on estimates).

Furthermore, Lundholm (1999) notes that, under current GAAP, accrual estimates for a given period are rarely compared with subsequent realizations, and managers, therefore, do not face the consequences of serious intentional or unintentional misestimates. To enhance the reliability of estimates, Lundholm proposes a requirement for an ex post report on the accuracy of prior estimates. It is highly likely that the specter of investors and board members focusing on large and consistent estimation errors (for example, the warranty provision was below the actual warranty costs in every quarter of the last 2 years) will provide managers with strong ex ante disincentives to manipulate the estimates and motivate them to spend more resources on improving the reliability of the estimation procedures. Finally, Lev and Zarowin (1999) and Lev et al. (2008) take the reconciliation of estimates with ex post realizations a step further by proposing that in case of large discrepancies, previous financial reports should be revised like the routine revisions of macro-economic data. Such a revision will improve the historical record of financial information, which has been shown to affect investors’ decisions (for example, Barth et al. 1999).